2023-07-13 05:59:38,846 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.quotas.TestSpaceQuotasWithSnapshots timeout: 13 mins 2023-07-13 05:59:39,033 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/74b62a1d-1af8-4c02-70de-8b0007eeebf7 2023-07-13 05:59:39,046 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=1, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-07-13 05:59:39,046 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/74b62a1d-1af8-4c02-70de-8b0007eeebf7/cluster_3a5fcc1f-2335-da3f-c526-d77046317035, deleteOnExit=true 2023-07-13 05:59:39,046 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-07-13 05:59:39,047 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/74b62a1d-1af8-4c02-70de-8b0007eeebf7/test.cache.data in system properties and HBase conf 2023-07-13 05:59:39,047 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/74b62a1d-1af8-4c02-70de-8b0007eeebf7/hadoop.tmp.dir in system properties and HBase conf 2023-07-13 05:59:39,048 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/74b62a1d-1af8-4c02-70de-8b0007eeebf7/hadoop.log.dir in system properties and HBase conf 2023-07-13 05:59:39,048 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/74b62a1d-1af8-4c02-70de-8b0007eeebf7/mapreduce.cluster.local.dir in system properties and HBase conf 2023-07-13 05:59:39,049 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/74b62a1d-1af8-4c02-70de-8b0007eeebf7/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-07-13 05:59:39,049 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-07-13 05:59:39,146 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-07-13 05:59:39,475 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-07-13 05:59:39,479 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/74b62a1d-1af8-4c02-70de-8b0007eeebf7/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-07-13 05:59:39,479 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/74b62a1d-1af8-4c02-70de-8b0007eeebf7/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-07-13 05:59:39,480 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/74b62a1d-1af8-4c02-70de-8b0007eeebf7/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-07-13 05:59:39,480 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/74b62a1d-1af8-4c02-70de-8b0007eeebf7/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-13 05:59:39,480 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/74b62a1d-1af8-4c02-70de-8b0007eeebf7/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-07-13 05:59:39,480 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/74b62a1d-1af8-4c02-70de-8b0007eeebf7/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-07-13 05:59:39,481 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/74b62a1d-1af8-4c02-70de-8b0007eeebf7/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-13 05:59:39,481 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/74b62a1d-1af8-4c02-70de-8b0007eeebf7/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-13 05:59:39,481 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/74b62a1d-1af8-4c02-70de-8b0007eeebf7/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-07-13 05:59:39,482 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/74b62a1d-1af8-4c02-70de-8b0007eeebf7/nfs.dump.dir in system properties and HBase conf 2023-07-13 05:59:39,482 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/74b62a1d-1af8-4c02-70de-8b0007eeebf7/java.io.tmpdir in system properties and HBase conf 2023-07-13 05:59:39,483 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/74b62a1d-1af8-4c02-70de-8b0007eeebf7/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-13 05:59:39,483 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/74b62a1d-1af8-4c02-70de-8b0007eeebf7/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-07-13 05:59:39,483 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/74b62a1d-1af8-4c02-70de-8b0007eeebf7/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-07-13 05:59:39,901 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-13 05:59:39,905 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-13 05:59:40,137 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-07-13 05:59:40,304 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-07-13 05:59:40,317 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-13 05:59:40,351 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-07-13 05:59:40,379 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/74b62a1d-1af8-4c02-70de-8b0007eeebf7/java.io.tmpdir/Jetty_localhost_localdomain_36467_hdfs____bnr0s/webapp 2023-07-13 05:59:40,510 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:36467 2023-07-13 05:59:40,517 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-13 05:59:40,517 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-13 05:59:40,936 WARN [Listener at localhost.localdomain/36315] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-13 05:59:41,084 WARN [Listener at localhost.localdomain/36315] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-13 05:59:41,104 WARN [Listener at localhost.localdomain/36315] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-13 05:59:41,112 INFO [Listener at localhost.localdomain/36315] log.Slf4jLog(67): jetty-6.1.26 2023-07-13 05:59:41,118 INFO [Listener at localhost.localdomain/36315] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/74b62a1d-1af8-4c02-70de-8b0007eeebf7/java.io.tmpdir/Jetty_localhost_42705_datanode____4gd1yd/webapp 2023-07-13 05:59:41,210 INFO [Listener at localhost.localdomain/36315] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42705 2023-07-13 05:59:41,554 WARN [Listener at localhost.localdomain/45717] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-13 05:59:41,907 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x6679268d493a5cb4: Processing first storage report for DS-0b01ad4e-a191-42d4-90fc-9736f1ea03fb from datanode 7f8df910-a3c0-47a0-8fe6-894f5b42ccb2 2023-07-13 05:59:41,908 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x6679268d493a5cb4: from storage DS-0b01ad4e-a191-42d4-90fc-9736f1ea03fb node DatanodeRegistration(127.0.0.1:38033, datanodeUuid=7f8df910-a3c0-47a0-8fe6-894f5b42ccb2, infoPort=40921, infoSecurePort=0, ipcPort=45717, storageInfo=lv=-57;cid=testClusterID;nsid=655826626;c=1689227979967), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-07-13 05:59:41,909 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x6679268d493a5cb4: Processing first storage report for DS-c24e1b64-05fb-424f-834d-7740e57a40eb from datanode 7f8df910-a3c0-47a0-8fe6-894f5b42ccb2 2023-07-13 05:59:41,909 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x6679268d493a5cb4: from storage DS-c24e1b64-05fb-424f-834d-7740e57a40eb node DatanodeRegistration(127.0.0.1:38033, datanodeUuid=7f8df910-a3c0-47a0-8fe6-894f5b42ccb2, infoPort=40921, infoSecurePort=0, ipcPort=45717, storageInfo=lv=-57;cid=testClusterID;nsid=655826626;c=1689227979967), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-13 05:59:41,930 DEBUG [Listener at localhost.localdomain/45717] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/74b62a1d-1af8-4c02-70de-8b0007eeebf7 2023-07-13 05:59:41,987 INFO [Listener at localhost.localdomain/45717] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/74b62a1d-1af8-4c02-70de-8b0007eeebf7/cluster_3a5fcc1f-2335-da3f-c526-d77046317035/zookeeper_0, clientPort=55963, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/74b62a1d-1af8-4c02-70de-8b0007eeebf7/cluster_3a5fcc1f-2335-da3f-c526-d77046317035/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/74b62a1d-1af8-4c02-70de-8b0007eeebf7/cluster_3a5fcc1f-2335-da3f-c526-d77046317035/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-07-13 05:59:41,998 INFO [Listener at localhost.localdomain/45717] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=55963 2023-07-13 05:59:42,006 INFO [Listener at localhost.localdomain/45717] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-13 05:59:42,008 INFO [Listener at localhost.localdomain/45717] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-13 05:59:42,640 INFO [Listener at localhost.localdomain/45717] util.FSUtils(471): Created version file at hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729 with version=8 2023-07-13 05:59:42,640 INFO [Listener at localhost.localdomain/45717] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/hbase-staging 2023-07-13 05:59:42,904 INFO [Listener at localhost.localdomain/45717] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-07-13 05:59:43,260 INFO [Listener at localhost.localdomain/45717] client.ConnectionUtils(127): master/jenkins-hbase20:0 server-side Connection retries=45 2023-07-13 05:59:43,284 INFO [Listener at localhost.localdomain/45717] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-13 05:59:43,284 INFO [Listener at localhost.localdomain/45717] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-13 05:59:43,284 INFO [Listener at localhost.localdomain/45717] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-13 05:59:43,284 INFO [Listener at localhost.localdomain/45717] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-13 05:59:43,285 INFO [Listener at localhost.localdomain/45717] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-13 05:59:43,396 INFO [Listener at localhost.localdomain/45717] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-13 05:59:43,456 DEBUG [Listener at localhost.localdomain/45717] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-07-13 05:59:43,534 INFO [Listener at localhost.localdomain/45717] ipc.NettyRpcServer(120): Bind to /148.251.75.209:34737 2023-07-13 05:59:43,543 INFO [Listener at localhost.localdomain/45717] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-13 05:59:43,545 INFO [Listener at localhost.localdomain/45717] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-13 05:59:43,562 INFO [Listener at localhost.localdomain/45717] zookeeper.RecoverableZooKeeper(93): Process identifier=master:34737 connecting to ZooKeeper ensemble=127.0.0.1:55963 2023-07-13 05:59:43,594 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:347370x0, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-13 05:59:43,597 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:34737-0x10084e9f1160000 connected 2023-07-13 05:59:43,616 DEBUG [Listener at localhost.localdomain/45717] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-13 05:59:43,617 DEBUG [Listener at localhost.localdomain/45717] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-13 05:59:43,621 DEBUG [Listener at localhost.localdomain/45717] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-13 05:59:43,630 DEBUG [Listener at localhost.localdomain/45717] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34737 2023-07-13 05:59:43,630 DEBUG [Listener at localhost.localdomain/45717] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34737 2023-07-13 05:59:43,631 DEBUG [Listener at localhost.localdomain/45717] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34737 2023-07-13 05:59:43,631 DEBUG [Listener at localhost.localdomain/45717] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34737 2023-07-13 05:59:43,632 DEBUG [Listener at localhost.localdomain/45717] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34737 2023-07-13 05:59:43,637 INFO [Listener at localhost.localdomain/45717] master.HMaster(444): hbase.rootdir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729, hbase.cluster.distributed=false 2023-07-13 05:59:43,701 INFO [Listener at localhost.localdomain/45717] client.ConnectionUtils(127): regionserver/jenkins-hbase20:0 server-side Connection retries=45 2023-07-13 05:59:43,701 INFO [Listener at localhost.localdomain/45717] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-13 05:59:43,702 INFO [Listener at localhost.localdomain/45717] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-13 05:59:43,702 INFO [Listener at localhost.localdomain/45717] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-13 05:59:43,702 INFO [Listener at localhost.localdomain/45717] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-13 05:59:43,702 INFO [Listener at localhost.localdomain/45717] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-13 05:59:43,706 INFO [Listener at localhost.localdomain/45717] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-13 05:59:43,708 INFO [Listener at localhost.localdomain/45717] ipc.NettyRpcServer(120): Bind to /148.251.75.209:35187 2023-07-13 05:59:43,710 INFO [Listener at localhost.localdomain/45717] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-13 05:59:43,715 DEBUG [Listener at localhost.localdomain/45717] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-13 05:59:43,716 INFO [Listener at localhost.localdomain/45717] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-13 05:59:43,718 INFO [Listener at localhost.localdomain/45717] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-13 05:59:43,720 INFO [Listener at localhost.localdomain/45717] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:35187 connecting to ZooKeeper ensemble=127.0.0.1:55963 2023-07-13 05:59:43,723 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:351870x0, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-13 05:59:43,724 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:35187-0x10084e9f1160001 connected 2023-07-13 05:59:43,725 DEBUG [Listener at localhost.localdomain/45717] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-13 05:59:43,726 DEBUG [Listener at localhost.localdomain/45717] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-13 05:59:43,726 DEBUG [Listener at localhost.localdomain/45717] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-13 05:59:43,727 DEBUG [Listener at localhost.localdomain/45717] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=35187 2023-07-13 05:59:43,727 DEBUG [Listener at localhost.localdomain/45717] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=35187 2023-07-13 05:59:43,728 DEBUG [Listener at localhost.localdomain/45717] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=35187 2023-07-13 05:59:43,728 DEBUG [Listener at localhost.localdomain/45717] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=35187 2023-07-13 05:59:43,729 DEBUG [Listener at localhost.localdomain/45717] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=35187 2023-07-13 05:59:43,730 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase20.apache.org,34737,1689227982756 2023-07-13 05:59:43,739 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-07-13 05:59:43,740 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase20.apache.org,34737,1689227982756 2023-07-13 05:59:43,756 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-13 05:59:43,756 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-13 05:59:43,757 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-13 05:59:43,757 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-13 05:59:43,758 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase20.apache.org,34737,1689227982756 from backup master directory 2023-07-13 05:59:43,759 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-13 05:59:43,763 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase20.apache.org,34737,1689227982756 2023-07-13 05:59:43,763 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-07-13 05:59:43,764 WARN [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-13 05:59:43,764 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase20.apache.org,34737,1689227982756 2023-07-13 05:59:43,766 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-07-13 05:59:43,768 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-07-13 05:59:43,850 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/hbase.id with ID: f7b3524f-aac2-4c0e-bbfd-401794046f67 2023-07-13 05:59:43,895 INFO [master/jenkins-hbase20:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-13 05:59:43,911 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-13 05:59:43,949 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x668f607f to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 05:59:43,976 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3434ef0a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 05:59:44,000 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-13 05:59:44,002 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-13 05:59:44,023 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-07-13 05:59:44,023 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-07-13 05:59:44,025 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-07-13 05:59:44,030 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-07-13 05:59:44,031 INFO [master/jenkins-hbase20:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-13 05:59:44,074 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/MasterData/data/master/store-tmp 2023-07-13 05:59:44,502 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 05:59:44,502 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-13 05:59:44,502 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-13 05:59:44,502 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-13 05:59:44,503 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-13 05:59:44,503 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-13 05:59:44,503 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-13 05:59:44,503 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-13 05:59:44,505 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/MasterData/WALs/jenkins-hbase20.apache.org,34737,1689227982756 2023-07-13 05:59:44,527 INFO [master/jenkins-hbase20:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C34737%2C1689227982756, suffix=, logDir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/MasterData/WALs/jenkins-hbase20.apache.org,34737,1689227982756, archiveDir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/MasterData/oldWALs, maxLogs=10 2023-07-13 05:59:44,573 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38033,DS-0b01ad4e-a191-42d4-90fc-9736f1ea03fb,DISK] 2023-07-13 05:59:44,580 DEBUG [RS-EventLoopGroup-3-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-07-13 05:59:44,630 INFO [master/jenkins-hbase20:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/MasterData/WALs/jenkins-hbase20.apache.org,34737,1689227982756/jenkins-hbase20.apache.org%2C34737%2C1689227982756.1689227984536 2023-07-13 05:59:44,631 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38033,DS-0b01ad4e-a191-42d4-90fc-9736f1ea03fb,DISK]] 2023-07-13 05:59:44,631 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-13 05:59:44,632 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 05:59:44,634 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-13 05:59:44,635 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-13 05:59:44,684 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-13 05:59:44,691 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-13 05:59:44,712 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-13 05:59:44,723 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-13 05:59:44,729 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-13 05:59:44,731 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-13 05:59:44,747 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-13 05:59:44,751 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-13 05:59:44,753 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69889273, jitterRate=0.04143132269382477}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-13 05:59:44,753 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-13 05:59:44,754 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-13 05:59:44,773 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-13 05:59:44,773 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-13 05:59:44,775 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-13 05:59:44,777 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-13 05:59:44,806 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 28 msec 2023-07-13 05:59:44,806 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-13 05:59:44,831 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-07-13 05:59:44,836 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-07-13 05:59:44,858 INFO [master/jenkins-hbase20:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-13 05:59:44,861 INFO [master/jenkins-hbase20:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-13 05:59:44,863 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-07-13 05:59:44,869 INFO [master/jenkins-hbase20:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-13 05:59:44,872 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-07-13 05:59:44,875 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-13 05:59:44,876 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-07-13 05:59:44,877 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-07-13 05:59:44,889 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-07-13 05:59:44,894 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-13 05:59:44,894 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-13 05:59:44,894 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-13 05:59:44,895 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase20.apache.org,34737,1689227982756, sessionid=0x10084e9f1160000, setting cluster-up flag (Was=false) 2023-07-13 05:59:44,908 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-13 05:59:44,912 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-07-13 05:59:44,913 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase20.apache.org,34737,1689227982756 2023-07-13 05:59:44,917 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-13 05:59:44,921 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-07-13 05:59:44,922 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase20.apache.org,34737,1689227982756 2023-07-13 05:59:44,924 WARN [master/jenkins-hbase20:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.hbase-snapshot/.tmp 2023-07-13 05:59:44,933 INFO [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(951): ClusterId : f7b3524f-aac2-4c0e-bbfd-401794046f67 2023-07-13 05:59:44,937 DEBUG [RS:0;jenkins-hbase20:35187] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-13 05:59:44,942 DEBUG [RS:0;jenkins-hbase20:35187] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-13 05:59:44,943 DEBUG [RS:0;jenkins-hbase20:35187] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-13 05:59:44,946 DEBUG [RS:0;jenkins-hbase20:35187] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-13 05:59:44,947 DEBUG [RS:0;jenkins-hbase20:35187] zookeeper.ReadOnlyZKClient(139): Connect 0x288e5e7b to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 05:59:44,951 DEBUG [RS:0;jenkins-hbase20:35187] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2649b51a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 05:59:44,952 DEBUG [RS:0;jenkins-hbase20:35187] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2276350b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-07-13 05:59:44,972 DEBUG [RS:0;jenkins-hbase20:35187] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase20:35187 2023-07-13 05:59:44,975 INFO [RS:0;jenkins-hbase20:35187] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-13 05:59:44,975 INFO [RS:0;jenkins-hbase20:35187] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-13 05:59:44,975 INFO [master/jenkins-hbase20:0:becomeActiveMaster] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.quotas.MasterQuotasObserver loaded, priority=536870911. 2023-07-13 05:59:44,976 DEBUG [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(1022): About to register with Master. 2023-07-13 05:59:44,978 INFO [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase20.apache.org,34737,1689227982756 with isa=jenkins-hbase20.apache.org/148.251.75.209:35187, startcode=1689227983701 2023-07-13 05:59:44,992 DEBUG [RS:0;jenkins-hbase20:35187] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-13 05:59:45,043 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:58167, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-07-13 05:59:45,044 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-07-13 05:59:45,051 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=34737] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2832) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:579) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-13 05:59:45,054 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-07-13 05:59:45,055 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-07-13 05:59:45,055 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-07-13 05:59:45,055 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-07-13 05:59:45,055 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase20:0, corePoolSize=10, maxPoolSize=10 2023-07-13 05:59:45,055 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-13 05:59:45,055 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=2, maxPoolSize=2 2023-07-13 05:59:45,055 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-13 05:59:45,056 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1689228015056 2023-07-13 05:59:45,059 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-13 05:59:45,063 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-07-13 05:59:45,064 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-07-13 05:59:45,069 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-13 05:59:45,070 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-13 05:59:45,077 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-13 05:59:45,077 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-13 05:59:45,078 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-13 05:59:45,078 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-13 05:59:45,078 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-13 05:59:45,080 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-13 05:59:45,081 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-13 05:59:45,081 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-13 05:59:45,083 DEBUG [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(2830): Master is not running yet 2023-07-13 05:59:45,083 WARN [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(1030): reportForDuty failed; sleeping 100 ms and then retrying. 2023-07-13 05:59:45,083 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-13 05:59:45,084 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-13 05:59:45,085 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.large.0-1689227985085,5,FailOnTimeoutGroup] 2023-07-13 05:59:45,085 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.small.0-1689227985085,5,FailOnTimeoutGroup] 2023-07-13 05:59:45,085 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-13 05:59:45,085 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-13 05:59:45,087 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-13 05:59:45,087 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-13 05:59:45,185 INFO [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase20.apache.org,34737,1689227982756 with isa=jenkins-hbase20.apache.org/148.251.75.209:35187, startcode=1689227983701 2023-07-13 05:59:45,195 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=34737] master.ServerManager(394): Registering regionserver=jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:45,200 DEBUG [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729 2023-07-13 05:59:45,200 DEBUG [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost.localdomain:36315 2023-07-13 05:59:45,200 DEBUG [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-13 05:59:45,205 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-13 05:59:45,205 DEBUG [RS:0;jenkins-hbase20:35187] zookeeper.ZKUtil(162): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:45,206 WARN [RS:0;jenkins-hbase20:35187] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-13 05:59:45,206 INFO [RS:0;jenkins-hbase20:35187] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-13 05:59:45,206 DEBUG [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(1948): logDir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/WALs/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:45,207 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase20.apache.org,35187,1689227983701] 2023-07-13 05:59:45,216 DEBUG [RS:0;jenkins-hbase20:35187] zookeeper.ZKUtil(162): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:45,227 DEBUG [RS:0;jenkins-hbase20:35187] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-13 05:59:45,233 INFO [RS:0;jenkins-hbase20:35187] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-13 05:59:45,249 INFO [RS:0;jenkins-hbase20:35187] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-13 05:59:45,252 INFO [RS:0;jenkins-hbase20:35187] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-13 05:59:45,252 INFO [RS:0;jenkins-hbase20:35187] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-13 05:59:45,253 INFO [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-13 05:59:45,258 INFO [RS:0;jenkins-hbase20:35187] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=5000, unit=MILLISECONDS is enabled. 2023-07-13 05:59:45,259 DEBUG [RS:0;jenkins-hbase20:35187] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-13 05:59:45,259 DEBUG [RS:0;jenkins-hbase20:35187] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-13 05:59:45,259 DEBUG [RS:0;jenkins-hbase20:35187] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-13 05:59:45,259 DEBUG [RS:0;jenkins-hbase20:35187] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-13 05:59:45,259 DEBUG [RS:0;jenkins-hbase20:35187] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-13 05:59:45,259 DEBUG [RS:0;jenkins-hbase20:35187] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase20:0, corePoolSize=2, maxPoolSize=2 2023-07-13 05:59:45,259 DEBUG [RS:0;jenkins-hbase20:35187] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-13 05:59:45,260 DEBUG [RS:0;jenkins-hbase20:35187] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-13 05:59:45,260 DEBUG [RS:0;jenkins-hbase20:35187] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-13 05:59:45,260 DEBUG [RS:0;jenkins-hbase20:35187] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-07-13 05:59:45,261 INFO [RS:0;jenkins-hbase20:35187] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-13 05:59:45,261 INFO [RS:0;jenkins-hbase20:35187] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-13 05:59:45,261 INFO [RS:0;jenkins-hbase20:35187] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-13 05:59:45,261 INFO [RS:0;jenkins-hbase20:35187] hbase.ChoreService(166): Chore ScheduledChore name=FileSystemUtilizationChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-13 05:59:45,273 INFO [RS:0;jenkins-hbase20:35187] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-13 05:59:45,274 INFO [RS:0;jenkins-hbase20:35187] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,35187,1689227983701-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-13 05:59:45,287 INFO [RS:0;jenkins-hbase20:35187] regionserver.Replication(203): jenkins-hbase20.apache.org,35187,1689227983701 started 2023-07-13 05:59:45,288 INFO [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(1637): Serving as jenkins-hbase20.apache.org,35187,1689227983701, RpcServer on jenkins-hbase20.apache.org/148.251.75.209:35187, sessionid=0x10084e9f1160001 2023-07-13 05:59:45,288 DEBUG [RS:0;jenkins-hbase20:35187] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-13 05:59:45,288 DEBUG [RS:0;jenkins-hbase20:35187] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:45,288 DEBUG [RS:0;jenkins-hbase20:35187] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,35187,1689227983701' 2023-07-13 05:59:45,288 DEBUG [RS:0;jenkins-hbase20:35187] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-13 05:59:45,289 DEBUG [RS:0;jenkins-hbase20:35187] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-13 05:59:45,289 DEBUG [RS:0;jenkins-hbase20:35187] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-13 05:59:45,289 DEBUG [RS:0;jenkins-hbase20:35187] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-13 05:59:45,289 DEBUG [RS:0;jenkins-hbase20:35187] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:45,289 DEBUG [RS:0;jenkins-hbase20:35187] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,35187,1689227983701' 2023-07-13 05:59:45,290 DEBUG [RS:0;jenkins-hbase20:35187] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-13 05:59:45,290 DEBUG [RS:0;jenkins-hbase20:35187] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-13 05:59:45,290 DEBUG [RS:0;jenkins-hbase20:35187] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-13 05:59:45,290 INFO [RS:0;jenkins-hbase20:35187] quotas.RegionServerRpcQuotaManager(67): Initializing RPC quota support 2023-07-13 05:59:45,292 INFO [RS:0;jenkins-hbase20:35187] hbase.ChoreService(166): Chore ScheduledChore name=QuotaRefresherChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-13 05:59:45,292 DEBUG [RS:0;jenkins-hbase20:35187] zookeeper.ZKUtil(398): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Unable to get data of znode /hbase/rpc-throttle because node does not exist (not an error) 2023-07-13 05:59:45,292 INFO [RS:0;jenkins-hbase20:35187] quotas.RegionServerRpcQuotaManager(73): Start rpc quota manager and rpc throttle enabled is true 2023-07-13 05:59:45,293 INFO [RS:0;jenkins-hbase20:35187] hbase.ChoreService(166): Chore ScheduledChore name=SpaceQuotaRefresherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-13 05:59:45,293 INFO [RS:0;jenkins-hbase20:35187] hbase.ChoreService(166): Chore ScheduledChore name=RegionSizeReportingChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-13 05:59:45,410 INFO [RS:0;jenkins-hbase20:35187] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C35187%2C1689227983701, suffix=, logDir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/WALs/jenkins-hbase20.apache.org,35187,1689227983701, archiveDir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/oldWALs, maxLogs=32 2023-07-13 05:59:45,425 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38033,DS-0b01ad4e-a191-42d4-90fc-9736f1ea03fb,DISK] 2023-07-13 05:59:45,428 INFO [RS:0;jenkins-hbase20:35187] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/WALs/jenkins-hbase20.apache.org,35187,1689227983701/jenkins-hbase20.apache.org%2C35187%2C1689227983701.1689227985412 2023-07-13 05:59:45,429 DEBUG [RS:0;jenkins-hbase20:35187] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38033,DS-0b01ad4e-a191-42d4-90fc-9736f1ea03fb,DISK]] 2023-07-13 05:59:45,500 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-13 05:59:45,502 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-13 05:59:45,503 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729 2023-07-13 05:59:45,921 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 05:59:45,926 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-13 05:59:45,930 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/meta/1588230740/info 2023-07-13 05:59:45,931 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-13 05:59:45,932 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-13 05:59:45,932 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-13 05:59:45,936 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/meta/1588230740/rep_barrier 2023-07-13 05:59:45,936 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-13 05:59:45,938 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-13 05:59:45,938 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-13 05:59:45,941 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/meta/1588230740/table 2023-07-13 05:59:45,942 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-13 05:59:45,943 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-13 05:59:45,944 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/meta/1588230740 2023-07-13 05:59:45,945 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/meta/1588230740 2023-07-13 05:59:45,948 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-13 05:59:45,950 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-13 05:59:45,953 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-13 05:59:45,954 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68439433, jitterRate=0.019827023148536682}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-13 05:59:45,954 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-13 05:59:45,954 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-13 05:59:45,955 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-13 05:59:45,955 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-13 05:59:45,955 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-13 05:59:45,955 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-13 05:59:45,956 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-13 05:59:45,956 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-13 05:59:45,962 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-07-13 05:59:45,962 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-07-13 05:59:45,970 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-13 05:59:45,980 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-13 05:59:45,983 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-07-13 05:59:46,137 DEBUG [jenkins-hbase20:34737] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-07-13 05:59:46,144 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase20.apache.org,35187,1689227983701, state=OPENING 2023-07-13 05:59:46,165 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-07-13 05:59:46,167 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-13 05:59:46,168 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-13 05:59:46,171 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase20.apache.org,35187,1689227983701}] 2023-07-13 05:59:46,360 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:46,363 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-13 05:59:46,367 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:57154, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-13 05:59:46,379 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-13 05:59:46,379 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-13 05:59:46,383 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C35187%2C1689227983701.meta, suffix=.meta, logDir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/WALs/jenkins-hbase20.apache.org,35187,1689227983701, archiveDir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/oldWALs, maxLogs=32 2023-07-13 05:59:46,402 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38033,DS-0b01ad4e-a191-42d4-90fc-9736f1ea03fb,DISK] 2023-07-13 05:59:46,406 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/WALs/jenkins-hbase20.apache.org,35187,1689227983701/jenkins-hbase20.apache.org%2C35187%2C1689227983701.meta.1689227986385.meta 2023-07-13 05:59:46,407 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38033,DS-0b01ad4e-a191-42d4-90fc-9736f1ea03fb,DISK]] 2023-07-13 05:59:46,408 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-13 05:59:46,410 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-13 05:59:46,428 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-13 05:59:46,432 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-13 05:59:46,437 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-13 05:59:46,437 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 05:59:46,437 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-13 05:59:46,437 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-13 05:59:46,440 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-13 05:59:46,441 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/meta/1588230740/info 2023-07-13 05:59:46,442 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/meta/1588230740/info 2023-07-13 05:59:46,442 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-13 05:59:46,443 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-13 05:59:46,443 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-13 05:59:46,444 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/meta/1588230740/rep_barrier 2023-07-13 05:59:46,444 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/meta/1588230740/rep_barrier 2023-07-13 05:59:46,445 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-13 05:59:46,445 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-13 05:59:46,446 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-13 05:59:46,447 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/meta/1588230740/table 2023-07-13 05:59:46,447 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/meta/1588230740/table 2023-07-13 05:59:46,447 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-13 05:59:46,448 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-13 05:59:46,450 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/meta/1588230740 2023-07-13 05:59:46,452 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/meta/1588230740 2023-07-13 05:59:46,455 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-13 05:59:46,457 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-13 05:59:46,458 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60664598, jitterRate=-0.09602704644203186}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-13 05:59:46,458 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-13 05:59:46,466 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1689227986355 2023-07-13 05:59:46,481 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-13 05:59:46,482 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-13 05:59:46,482 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase20.apache.org,35187,1689227983701, state=OPEN 2023-07-13 05:59:46,484 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-07-13 05:59:46,484 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-13 05:59:46,489 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-07-13 05:59:46,489 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase20.apache.org,35187,1689227983701 in 313 msec 2023-07-13 05:59:46,495 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-07-13 05:59:46,495 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 521 msec 2023-07-13 05:59:46,500 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 1.5160 sec 2023-07-13 05:59:46,500 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1689227986500, completionTime=-1 2023-07-13 05:59:46,501 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-07-13 05:59:46,501 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-13 05:59:46,554 DEBUG [hconnection-0x55e0d78d-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 05:59:46,557 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:57162, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 05:59:46,573 INFO [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=1 2023-07-13 05:59:46,573 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1689228046573 2023-07-13 05:59:46,573 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1689228106573 2023-07-13 05:59:46,573 INFO [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 72 msec 2023-07-13 05:59:46,592 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,34737,1689227982756-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-13 05:59:46,592 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,34737,1689227982756-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-13 05:59:46,592 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,34737,1689227982756-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-13 05:59:46,594 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase20:34737, period=300000, unit=MILLISECONDS is enabled. 2023-07-13 05:59:46,594 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-13 05:59:46,599 DEBUG [master/jenkins-hbase20:0.Chore.1] janitor.CatalogJanitor(175): 2023-07-13 05:59:46,606 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-07-13 05:59:46,608 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-13 05:59:46,615 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-07-13 05:59:46,618 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-07-13 05:59:46,621 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-13 05:59:46,642 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/hbase/namespace/d0f4e6ea3f932d1de6f1c8e1ab57b30a 2023-07-13 05:59:46,645 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/hbase/namespace/d0f4e6ea3f932d1de6f1c8e1ab57b30a empty. 2023-07-13 05:59:46,646 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/hbase/namespace/d0f4e6ea3f932d1de6f1c8e1ab57b30a 2023-07-13 05:59:46,646 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-07-13 05:59:47,074 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-07-13 05:59:47,078 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => d0f4e6ea3f932d1de6f1c8e1ab57b30a, NAME => 'hbase:namespace,,1689227986607.d0f4e6ea3f932d1de6f1c8e1ab57b30a.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp 2023-07-13 05:59:47,312 INFO [regionserver/jenkins-hbase20:0.Chore.1] quotas.SpaceQuotaRefresherChore(80): Quota table not found, skipping quota manager cache refresh. 2023-07-13 05:59:47,317 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.PleaseHoldException: Master is initializing at org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2840) at org.apache.hadoop.hbase.master.MasterRpcServices.reportRegionSpaceUse(MasterRpcServices.java:2272) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15962) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-13 05:59:47,318 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] ipc.CallRunner(144): callId: 21 service: RegionServerStatusService methodName: ReportRegionSpaceUse size: 66 connection: 148.251.75.209:58167 deadline: 1689227997317, exception=org.apache.hadoop.hbase.PleaseHoldException: Master is initializing 2023-07-13 05:59:47,496 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689227986607.d0f4e6ea3f932d1de6f1c8e1ab57b30a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 05:59:47,496 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing d0f4e6ea3f932d1de6f1c8e1ab57b30a, disabling compactions & flushes 2023-07-13 05:59:47,496 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689227986607.d0f4e6ea3f932d1de6f1c8e1ab57b30a. 2023-07-13 05:59:47,496 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689227986607.d0f4e6ea3f932d1de6f1c8e1ab57b30a. 2023-07-13 05:59:47,496 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689227986607.d0f4e6ea3f932d1de6f1c8e1ab57b30a. after waiting 0 ms 2023-07-13 05:59:47,496 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689227986607.d0f4e6ea3f932d1de6f1c8e1ab57b30a. 2023-07-13 05:59:47,496 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689227986607.d0f4e6ea3f932d1de6f1c8e1ab57b30a. 2023-07-13 05:59:47,496 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for d0f4e6ea3f932d1de6f1c8e1ab57b30a: 2023-07-13 05:59:47,503 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-07-13 05:59:47,521 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1689227986607.d0f4e6ea3f932d1de6f1c8e1ab57b30a.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689227987507"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689227987507"}]},"ts":"1689227987507"} 2023-07-13 05:59:47,543 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-13 05:59:47,544 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-13 05:59:47,550 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689227987545"}]},"ts":"1689227987545"} 2023-07-13 05:59:47,554 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-07-13 05:59:47,560 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=d0f4e6ea3f932d1de6f1c8e1ab57b30a, ASSIGN}] 2023-07-13 05:59:47,563 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=d0f4e6ea3f932d1de6f1c8e1ab57b30a, ASSIGN 2023-07-13 05:59:47,564 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=d0f4e6ea3f932d1de6f1c8e1ab57b30a, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,35187,1689227983701; forceNewPlan=false, retain=false 2023-07-13 05:59:47,716 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=d0f4e6ea3f932d1de6f1c8e1ab57b30a, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:47,717 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1689227986607.d0f4e6ea3f932d1de6f1c8e1ab57b30a.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689227987716"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689227987716"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689227987716"}]},"ts":"1689227987716"} 2023-07-13 05:59:47,726 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure d0f4e6ea3f932d1de6f1c8e1ab57b30a, server=jenkins-hbase20.apache.org,35187,1689227983701}] 2023-07-13 05:59:47,893 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1689227986607.d0f4e6ea3f932d1de6f1c8e1ab57b30a. 2023-07-13 05:59:47,894 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d0f4e6ea3f932d1de6f1c8e1ab57b30a, NAME => 'hbase:namespace,,1689227986607.d0f4e6ea3f932d1de6f1c8e1ab57b30a.', STARTKEY => '', ENDKEY => ''} 2023-07-13 05:59:47,896 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace d0f4e6ea3f932d1de6f1c8e1ab57b30a 2023-07-13 05:59:47,897 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1689227986607.d0f4e6ea3f932d1de6f1c8e1ab57b30a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 05:59:47,897 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for d0f4e6ea3f932d1de6f1c8e1ab57b30a 2023-07-13 05:59:47,897 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for d0f4e6ea3f932d1de6f1c8e1ab57b30a 2023-07-13 05:59:47,900 INFO [StoreOpener-d0f4e6ea3f932d1de6f1c8e1ab57b30a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region d0f4e6ea3f932d1de6f1c8e1ab57b30a 2023-07-13 05:59:47,902 DEBUG [StoreOpener-d0f4e6ea3f932d1de6f1c8e1ab57b30a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/namespace/d0f4e6ea3f932d1de6f1c8e1ab57b30a/info 2023-07-13 05:59:47,902 DEBUG [StoreOpener-d0f4e6ea3f932d1de6f1c8e1ab57b30a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/namespace/d0f4e6ea3f932d1de6f1c8e1ab57b30a/info 2023-07-13 05:59:47,902 INFO [StoreOpener-d0f4e6ea3f932d1de6f1c8e1ab57b30a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d0f4e6ea3f932d1de6f1c8e1ab57b30a columnFamilyName info 2023-07-13 05:59:47,903 INFO [StoreOpener-d0f4e6ea3f932d1de6f1c8e1ab57b30a-1] regionserver.HStore(310): Store=d0f4e6ea3f932d1de6f1c8e1ab57b30a/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-13 05:59:47,904 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/namespace/d0f4e6ea3f932d1de6f1c8e1ab57b30a 2023-07-13 05:59:47,905 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/namespace/d0f4e6ea3f932d1de6f1c8e1ab57b30a 2023-07-13 05:59:47,909 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for d0f4e6ea3f932d1de6f1c8e1ab57b30a 2023-07-13 05:59:47,913 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/namespace/d0f4e6ea3f932d1de6f1c8e1ab57b30a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-13 05:59:47,914 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened d0f4e6ea3f932d1de6f1c8e1ab57b30a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60907667, jitterRate=-0.0924050360918045}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-13 05:59:47,914 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for d0f4e6ea3f932d1de6f1c8e1ab57b30a: 2023-07-13 05:59:47,916 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1689227986607.d0f4e6ea3f932d1de6f1c8e1ab57b30a., pid=6, masterSystemTime=1689227987882 2023-07-13 05:59:47,921 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1689227986607.d0f4e6ea3f932d1de6f1c8e1ab57b30a. 2023-07-13 05:59:47,921 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1689227986607.d0f4e6ea3f932d1de6f1c8e1ab57b30a. 2023-07-13 05:59:47,922 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=d0f4e6ea3f932d1de6f1c8e1ab57b30a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:47,923 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1689227986607.d0f4e6ea3f932d1de6f1c8e1ab57b30a.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1689227987922"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689227987922"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689227987922"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689227987922"}]},"ts":"1689227987922"} 2023-07-13 05:59:47,932 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-07-13 05:59:47,932 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure d0f4e6ea3f932d1de6f1c8e1ab57b30a, server=jenkins-hbase20.apache.org,35187,1689227983701 in 202 msec 2023-07-13 05:59:47,935 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-07-13 05:59:47,935 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=d0f4e6ea3f932d1de6f1c8e1ab57b30a, ASSIGN in 372 msec 2023-07-13 05:59:47,936 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-13 05:59:47,937 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689227987937"}]},"ts":"1689227987937"} 2023-07-13 05:59:47,940 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-07-13 05:59:47,943 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-07-13 05:59:47,945 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 1.3340 sec 2023-07-13 05:59:48,020 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-07-13 05:59:48,081 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-07-13 05:59:48,081 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-13 05:59:48,119 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-07-13 05:59:48,133 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-07-13 05:59:48,139 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 28 msec 2023-07-13 05:59:48,152 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-07-13 05:59:48,172 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-07-13 05:59:48,185 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 31 msec 2023-07-13 05:59:48,206 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-07-13 05:59:48,208 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-07-13 05:59:48,208 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 4.444sec 2023-07-13 05:59:48,210 INFO [master/jenkins-hbase20:0:becomeActiveMaster] quotas.MasterQuotaManager(103): Quota table not found. Creating... 2023-07-13 05:59:48,210 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:quota', {NAME => 'q', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'u', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-13 05:59:48,212 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:quota 2023-07-13 05:59:48,212 INFO [master/jenkins-hbase20:0:becomeActiveMaster] quotas.MasterQuotaManager(107): Initializing quota support 2023-07-13 05:59:48,214 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:quota execute state=CREATE_TABLE_PRE_OPERATION 2023-07-13 05:59:48,214 INFO [master/jenkins-hbase20:0:becomeActiveMaster] namespace.NamespaceStateManager(59): Namespace State Manager started. 2023-07-13 05:59:48,216 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:quota execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-13 05:59:48,218 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/hbase/quota/c8571aee40cfd11e8cbf1a696291b517 2023-07-13 05:59:48,220 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/hbase/quota/c8571aee40cfd11e8cbf1a696291b517 empty. 2023-07-13 05:59:48,221 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/hbase/quota/c8571aee40cfd11e8cbf1a696291b517 2023-07-13 05:59:48,221 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived hbase:quota regions 2023-07-13 05:59:48,222 INFO [master/jenkins-hbase20:0:becomeActiveMaster] namespace.NamespaceStateManager(222): Finished updating state of 2 namespaces. 2023-07-13 05:59:48,222 INFO [master/jenkins-hbase20:0:becomeActiveMaster] namespace.NamespaceAuditor(50): NamespaceAuditor started. 2023-07-13 05:59:48,224 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=QuotaObserverChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-13 05:59:48,225 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=QuotaObserverChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-13 05:59:48,227 INFO [master/jenkins-hbase20:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-13 05:59:48,227 INFO [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-13 05:59:48,228 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,34737,1689227982756-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-13 05:59:48,229 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,34737,1689227982756-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-13 05:59:48,240 DEBUG [Listener at localhost.localdomain/45717] zookeeper.ReadOnlyZKClient(139): Connect 0x404a8258 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 05:59:48,244 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-13 05:59:48,256 DEBUG [Listener at localhost.localdomain/45717] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6b9f1b8e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 05:59:48,268 DEBUG [hconnection-0xeb16c58-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 05:59:48,278 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:57176, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 05:59:48,280 INFO [Listener at localhost.localdomain/45717] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase20.apache.org,34737,1689227982756 2023-07-13 05:59:48,281 INFO [Listener at localhost.localdomain/45717] hbase.Waiter(180): Waiting up to [10,000] milli-secs(wait.for.ratio=[1]) 2023-07-13 05:59:48,300 INFO [regionserver/jenkins-hbase20:0.Chore.1] quotas.SpaceQuotaRefresherChore(80): Quota table not found, skipping quota manager cache refresh. 2023-07-13 05:59:48,640 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/hbase/quota/.tabledesc/.tableinfo.0000000001 2023-07-13 05:59:48,643 INFO [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(7675): creating {ENCODED => c8571aee40cfd11e8cbf1a696291b517, NAME => 'hbase:quota,,1689227988210.c8571aee40cfd11e8cbf1a696291b517.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:quota', {NAME => 'q', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'u', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp 2023-07-13 05:59:49,059 DEBUG [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(866): Instantiated hbase:quota,,1689227988210.c8571aee40cfd11e8cbf1a696291b517.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 05:59:49,059 DEBUG [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(1604): Closing c8571aee40cfd11e8cbf1a696291b517, disabling compactions & flushes 2023-07-13 05:59:49,059 INFO [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(1626): Closing region hbase:quota,,1689227988210.c8571aee40cfd11e8cbf1a696291b517. 2023-07-13 05:59:49,060 DEBUG [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:quota,,1689227988210.c8571aee40cfd11e8cbf1a696291b517. 2023-07-13 05:59:49,060 DEBUG [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:quota,,1689227988210.c8571aee40cfd11e8cbf1a696291b517. after waiting 0 ms 2023-07-13 05:59:49,060 DEBUG [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:quota,,1689227988210.c8571aee40cfd11e8cbf1a696291b517. 2023-07-13 05:59:49,060 INFO [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(1838): Closed hbase:quota,,1689227988210.c8571aee40cfd11e8cbf1a696291b517. 2023-07-13 05:59:49,060 DEBUG [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(1558): Region close journal for c8571aee40cfd11e8cbf1a696291b517: 2023-07-13 05:59:49,065 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:quota execute state=CREATE_TABLE_ADD_TO_META 2023-07-13 05:59:49,066 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:quota,,1689227988210.c8571aee40cfd11e8cbf1a696291b517.","families":{"info":[{"qualifier":"regioninfo","vlen":37,"tag":[],"timestamp":"1689227989066"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689227989066"}]},"ts":"1689227989066"} 2023-07-13 05:59:49,069 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-13 05:59:49,070 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:quota execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-13 05:59:49,071 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:quota","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689227989070"}]},"ts":"1689227989070"} 2023-07-13 05:59:49,073 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=hbase:quota, state=ENABLING in hbase:meta 2023-07-13 05:59:49,076 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:quota, region=c8571aee40cfd11e8cbf1a696291b517, ASSIGN}] 2023-07-13 05:59:49,079 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:quota, region=c8571aee40cfd11e8cbf1a696291b517, ASSIGN 2023-07-13 05:59:49,082 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:quota, region=c8571aee40cfd11e8cbf1a696291b517, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,35187,1689227983701; forceNewPlan=false, retain=false 2023-07-13 05:59:49,127 INFO [Listener at localhost.localdomain/45717] hbase.ResourceChecker(147): before: quotas.TestSpaceQuotasWithSnapshots#testRematerializedTablesDoNoInheritSpace Thread=211, OpenFileDescriptor=473, MaxFileDescriptor=60000, SystemLoadAverage=178, ProcessCount=165, AvailableMemoryMB=6911 2023-07-13 05:59:49,140 DEBUG [Listener at localhost.localdomain/45717] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-07-13 05:59:49,143 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:36036, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-07-13 05:59:49,155 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.HMaster$4(2112): Client=jenkins//148.251.75.209 create 'testRematerializedTablesDoNoInheritSpace0', {NAME => 'f1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-13 05:59:49,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] procedure2.ProcedureExecutor(1029): Stored pid=11, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:49,159 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=11, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRematerializedTablesDoNoInheritSpace0 execute state=CREATE_TABLE_PRE_OPERATION 2023-07-13 05:59:49,161 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=11, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRematerializedTablesDoNoInheritSpace0 execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-13 05:59:49,164 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(700): Client=jenkins//148.251.75.209 procedure request for creating table: namespace: "default" qualifier: "testRematerializedTablesDoNoInheritSpace0" procId is: 11 2023-07-13 05:59:49,164 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testRematerializedTablesDoNoInheritSpace0/c96fda412d307d645bd66b267f4787dd 2023-07-13 05:59:49,165 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testRematerializedTablesDoNoInheritSpace0/c96fda412d307d645bd66b267f4787dd empty. 2023-07-13 05:59:49,166 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testRematerializedTablesDoNoInheritSpace0/c96fda412d307d645bd66b267f4787dd 2023-07-13 05:59:49,166 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testRematerializedTablesDoNoInheritSpace0 regions 2023-07-13 05:59:49,174 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1230): Checking to see if procedure is done pid=11 2023-07-13 05:59:49,186 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testRematerializedTablesDoNoInheritSpace0/.tabledesc/.tableinfo.0000000001 2023-07-13 05:59:49,188 INFO [RegionOpenAndInit-testRematerializedTablesDoNoInheritSpace0-pool-0] regionserver.HRegion(7675): creating {ENCODED => c96fda412d307d645bd66b267f4787dd, NAME => 'testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRematerializedTablesDoNoInheritSpace0', {NAME => 'f1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp 2023-07-13 05:59:49,234 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=c8571aee40cfd11e8cbf1a696291b517, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:49,234 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:quota,,1689227988210.c8571aee40cfd11e8cbf1a696291b517.","families":{"info":[{"qualifier":"regioninfo","vlen":37,"tag":[],"timestamp":"1689227989233"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689227989233"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689227989233"}]},"ts":"1689227989233"} 2023-07-13 05:59:49,237 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=12, ppid=10, state=RUNNABLE; OpenRegionProcedure c8571aee40cfd11e8cbf1a696291b517, server=jenkins-hbase20.apache.org,35187,1689227983701}] 2023-07-13 05:59:49,282 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1230): Checking to see if procedure is done pid=11 2023-07-13 05:59:49,302 DEBUG [regionserver/jenkins-hbase20:0.Chore.1] client.ConnectionImplementation(1060): locateRegionInMeta parentTable='hbase:meta', attempt=0 of 46 failed; retrying after sleep of 46 org.apache.hadoop.hbase.client.NoServerForRegionException: No server address listed in hbase:meta for region hbase:quota,,1689227988210.c8571aee40cfd11e8cbf1a696291b517. containing row t. at org.apache.hadoop.hbase.client.ConnectionImplementation.locateRegionInMeta(ConnectionImplementation.java:1029) at org.apache.hadoop.hbase.client.ConnectionImplementation.locateRegion(ConnectionImplementation.java:866) at org.apache.hadoop.hbase.client.ServerConnectionUtils$ShortCircuitingClusterConnection.locateRegion(ServerConnectionUtils.java:70) at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:319) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:157) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:53) at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithoutRetries(RpcRetryingCallerImpl.java:187) at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:266) at org.apache.hadoop.hbase.client.ClientScanner.loadCache(ClientScanner.java:435) at org.apache.hadoop.hbase.client.ClientScanner.nextWithSyncCache(ClientScanner.java:309) at org.apache.hadoop.hbase.client.ClientScanner.next(ClientScanner.java:594) at org.apache.hadoop.hbase.client.ResultScanner$1.hasNext(ResultScanner.java:51) at org.apache.hadoop.hbase.quotas.SpaceQuotaRefresherChore.fetchSnapshotsFromQuotaTable(SpaceQuotaRefresherChore.java:186) at org.apache.hadoop.hbase.quotas.SpaceQuotaRefresherChore.chore(SpaceQuotaRefresherChore.java:91) at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:107) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-07-13 05:59:49,402 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open hbase:quota,,1689227988210.c8571aee40cfd11e8cbf1a696291b517. 2023-07-13 05:59:49,403 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c8571aee40cfd11e8cbf1a696291b517, NAME => 'hbase:quota,,1689227988210.c8571aee40cfd11e8cbf1a696291b517.', STARTKEY => '', ENDKEY => ''} 2023-07-13 05:59:49,405 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table quota c8571aee40cfd11e8cbf1a696291b517 2023-07-13 05:59:49,405 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated hbase:quota,,1689227988210.c8571aee40cfd11e8cbf1a696291b517.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 05:59:49,405 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for c8571aee40cfd11e8cbf1a696291b517 2023-07-13 05:59:49,405 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for c8571aee40cfd11e8cbf1a696291b517 2023-07-13 05:59:49,407 DEBUG [regionserver/jenkins-hbase20:0.Chore.1] client.ConnectionImplementation(1060): locateRegionInMeta parentTable='hbase:meta', attempt=1 of 46 failed; retrying after sleep of 46 org.apache.hadoop.hbase.client.NoServerForRegionException: No server address listed in hbase:meta for region hbase:quota,,1689227988210.c8571aee40cfd11e8cbf1a696291b517. containing row t. at org.apache.hadoop.hbase.client.ConnectionImplementation.locateRegionInMeta(ConnectionImplementation.java:1029) at org.apache.hadoop.hbase.client.ConnectionImplementation.locateRegion(ConnectionImplementation.java:866) at org.apache.hadoop.hbase.client.ServerConnectionUtils$ShortCircuitingClusterConnection.locateRegion(ServerConnectionUtils.java:70) at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:319) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:157) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:53) at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithoutRetries(RpcRetryingCallerImpl.java:187) at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:266) at org.apache.hadoop.hbase.client.ClientScanner.loadCache(ClientScanner.java:435) at org.apache.hadoop.hbase.client.ClientScanner.nextWithSyncCache(ClientScanner.java:309) at org.apache.hadoop.hbase.client.ClientScanner.next(ClientScanner.java:594) at org.apache.hadoop.hbase.client.ResultScanner$1.hasNext(ResultScanner.java:51) at org.apache.hadoop.hbase.quotas.SpaceQuotaRefresherChore.fetchSnapshotsFromQuotaTable(SpaceQuotaRefresherChore.java:186) at org.apache.hadoop.hbase.quotas.SpaceQuotaRefresherChore.chore(SpaceQuotaRefresherChore.java:91) at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:107) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-07-13 05:59:49,408 INFO [StoreOpener-c8571aee40cfd11e8cbf1a696291b517-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family q of region c8571aee40cfd11e8cbf1a696291b517 2023-07-13 05:59:49,410 DEBUG [StoreOpener-c8571aee40cfd11e8cbf1a696291b517-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/quota/c8571aee40cfd11e8cbf1a696291b517/q 2023-07-13 05:59:49,410 DEBUG [StoreOpener-c8571aee40cfd11e8cbf1a696291b517-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/quota/c8571aee40cfd11e8cbf1a696291b517/q 2023-07-13 05:59:49,411 INFO [StoreOpener-c8571aee40cfd11e8cbf1a696291b517-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c8571aee40cfd11e8cbf1a696291b517 columnFamilyName q 2023-07-13 05:59:49,412 INFO [StoreOpener-c8571aee40cfd11e8cbf1a696291b517-1] regionserver.HStore(310): Store=c8571aee40cfd11e8cbf1a696291b517/q, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-13 05:59:49,412 INFO [StoreOpener-c8571aee40cfd11e8cbf1a696291b517-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family u of region c8571aee40cfd11e8cbf1a696291b517 2023-07-13 05:59:49,414 DEBUG [StoreOpener-c8571aee40cfd11e8cbf1a696291b517-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/quota/c8571aee40cfd11e8cbf1a696291b517/u 2023-07-13 05:59:49,414 DEBUG [StoreOpener-c8571aee40cfd11e8cbf1a696291b517-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/quota/c8571aee40cfd11e8cbf1a696291b517/u 2023-07-13 05:59:49,415 INFO [StoreOpener-c8571aee40cfd11e8cbf1a696291b517-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c8571aee40cfd11e8cbf1a696291b517 columnFamilyName u 2023-07-13 05:59:49,415 INFO [StoreOpener-c8571aee40cfd11e8cbf1a696291b517-1] regionserver.HStore(310): Store=c8571aee40cfd11e8cbf1a696291b517/u, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-13 05:59:49,418 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/quota/c8571aee40cfd11e8cbf1a696291b517 2023-07-13 05:59:49,419 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/quota/c8571aee40cfd11e8cbf1a696291b517 2023-07-13 05:59:49,423 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:quota descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-13 05:59:49,426 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for c8571aee40cfd11e8cbf1a696291b517 2023-07-13 05:59:49,430 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/quota/c8571aee40cfd11e8cbf1a696291b517/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-13 05:59:49,432 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened c8571aee40cfd11e8cbf1a696291b517; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62370532, jitterRate=-0.07060664892196655}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-13 05:59:49,432 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for c8571aee40cfd11e8cbf1a696291b517: 2023-07-13 05:59:49,434 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:quota,,1689227988210.c8571aee40cfd11e8cbf1a696291b517., pid=12, masterSystemTime=1689227989395 2023-07-13 05:59:49,437 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:quota,,1689227988210.c8571aee40cfd11e8cbf1a696291b517. 2023-07-13 05:59:49,437 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened hbase:quota,,1689227988210.c8571aee40cfd11e8cbf1a696291b517. 2023-07-13 05:59:49,438 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=c8571aee40cfd11e8cbf1a696291b517, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:49,439 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:quota,,1689227988210.c8571aee40cfd11e8cbf1a696291b517.","families":{"info":[{"qualifier":"regioninfo","vlen":37,"tag":[],"timestamp":"1689227989438"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689227989438"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689227989438"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689227989438"}]},"ts":"1689227989438"} 2023-07-13 05:59:49,447 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=12, resume processing ppid=10 2023-07-13 05:59:49,447 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=12, ppid=10, state=SUCCESS; OpenRegionProcedure c8571aee40cfd11e8cbf1a696291b517, server=jenkins-hbase20.apache.org,35187,1689227983701 in 205 msec 2023-07-13 05:59:49,452 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-07-13 05:59:49,452 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=hbase:quota, region=c8571aee40cfd11e8cbf1a696291b517, ASSIGN in 371 msec 2023-07-13 05:59:49,453 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:quota execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-13 05:59:49,453 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:quota","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689227989453"}]},"ts":"1689227989453"} 2023-07-13 05:59:49,455 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:quota, state=ENABLED in hbase:meta 2023-07-13 05:59:49,459 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:quota execute state=CREATE_TABLE_POST_OPERATION 2023-07-13 05:59:49,463 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=hbase:quota in 1.2490 sec 2023-07-13 05:59:49,485 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1230): Checking to see if procedure is done pid=11 2023-07-13 05:59:49,608 DEBUG [RegionOpenAndInit-testRematerializedTablesDoNoInheritSpace0-pool-0] regionserver.HRegion(866): Instantiated testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 05:59:49,608 DEBUG [RegionOpenAndInit-testRematerializedTablesDoNoInheritSpace0-pool-0] regionserver.HRegion(1604): Closing c96fda412d307d645bd66b267f4787dd, disabling compactions & flushes 2023-07-13 05:59:49,608 INFO [RegionOpenAndInit-testRematerializedTablesDoNoInheritSpace0-pool-0] regionserver.HRegion(1626): Closing region testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd. 2023-07-13 05:59:49,608 DEBUG [RegionOpenAndInit-testRematerializedTablesDoNoInheritSpace0-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd. 2023-07-13 05:59:49,609 DEBUG [RegionOpenAndInit-testRematerializedTablesDoNoInheritSpace0-pool-0] regionserver.HRegion(1714): Acquired close lock on testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd. after waiting 0 ms 2023-07-13 05:59:49,609 DEBUG [RegionOpenAndInit-testRematerializedTablesDoNoInheritSpace0-pool-0] regionserver.HRegion(1724): Updates disabled for region testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd. 2023-07-13 05:59:49,609 INFO [RegionOpenAndInit-testRematerializedTablesDoNoInheritSpace0-pool-0] regionserver.HRegion(1838): Closed testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd. 2023-07-13 05:59:49,609 DEBUG [RegionOpenAndInit-testRematerializedTablesDoNoInheritSpace0-pool-0] regionserver.HRegion(1558): Region close journal for c96fda412d307d645bd66b267f4787dd: 2023-07-13 05:59:49,614 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=11, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRematerializedTablesDoNoInheritSpace0 execute state=CREATE_TABLE_ADD_TO_META 2023-07-13 05:59:49,616 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1689227989616"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689227989616"}]},"ts":"1689227989616"} 2023-07-13 05:59:49,618 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-13 05:59:49,620 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=11, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRematerializedTablesDoNoInheritSpace0 execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-13 05:59:49,620 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRematerializedTablesDoNoInheritSpace0","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689227989620"}]},"ts":"1689227989620"} 2023-07-13 05:59:49,623 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testRematerializedTablesDoNoInheritSpace0, state=ENABLING in hbase:meta 2023-07-13 05:59:49,626 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=11, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRematerializedTablesDoNoInheritSpace0, region=c96fda412d307d645bd66b267f4787dd, ASSIGN}] 2023-07-13 05:59:49,628 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=11, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRematerializedTablesDoNoInheritSpace0, region=c96fda412d307d645bd66b267f4787dd, ASSIGN 2023-07-13 05:59:49,630 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=13, ppid=11, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRematerializedTablesDoNoInheritSpace0, region=c96fda412d307d645bd66b267f4787dd, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,35187,1689227983701; forceNewPlan=false, retain=false 2023-07-13 05:59:49,781 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=c96fda412d307d645bd66b267f4787dd, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:49,781 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1689227989780"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689227989780"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689227989780"}]},"ts":"1689227989780"} 2023-07-13 05:59:49,784 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; OpenRegionProcedure c96fda412d307d645bd66b267f4787dd, server=jenkins-hbase20.apache.org,35187,1689227983701}] 2023-07-13 05:59:49,787 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1230): Checking to see if procedure is done pid=11 2023-07-13 05:59:49,950 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd. 2023-07-13 05:59:49,950 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c96fda412d307d645bd66b267f4787dd, NAME => 'testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd.', STARTKEY => '', ENDKEY => ''} 2023-07-13 05:59:49,951 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRematerializedTablesDoNoInheritSpace0 c96fda412d307d645bd66b267f4787dd 2023-07-13 05:59:49,951 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 05:59:49,951 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for c96fda412d307d645bd66b267f4787dd 2023-07-13 05:59:49,951 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for c96fda412d307d645bd66b267f4787dd 2023-07-13 05:59:49,953 INFO [StoreOpener-c96fda412d307d645bd66b267f4787dd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region c96fda412d307d645bd66b267f4787dd 2023-07-13 05:59:49,956 DEBUG [StoreOpener-c96fda412d307d645bd66b267f4787dd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testRematerializedTablesDoNoInheritSpace0/c96fda412d307d645bd66b267f4787dd/f1 2023-07-13 05:59:49,956 DEBUG [StoreOpener-c96fda412d307d645bd66b267f4787dd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testRematerializedTablesDoNoInheritSpace0/c96fda412d307d645bd66b267f4787dd/f1 2023-07-13 05:59:49,956 INFO [StoreOpener-c96fda412d307d645bd66b267f4787dd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c96fda412d307d645bd66b267f4787dd columnFamilyName f1 2023-07-13 05:59:49,957 INFO [StoreOpener-c96fda412d307d645bd66b267f4787dd-1] regionserver.HStore(310): Store=c96fda412d307d645bd66b267f4787dd/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-13 05:59:49,959 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testRematerializedTablesDoNoInheritSpace0/c96fda412d307d645bd66b267f4787dd 2023-07-13 05:59:49,961 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testRematerializedTablesDoNoInheritSpace0/c96fda412d307d645bd66b267f4787dd 2023-07-13 05:59:49,965 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for c96fda412d307d645bd66b267f4787dd 2023-07-13 05:59:49,968 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testRematerializedTablesDoNoInheritSpace0/c96fda412d307d645bd66b267f4787dd/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-13 05:59:49,969 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened c96fda412d307d645bd66b267f4787dd; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60354781, jitterRate=-0.10064367949962616}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-13 05:59:49,969 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for c96fda412d307d645bd66b267f4787dd: 2023-07-13 05:59:49,970 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd., pid=14, masterSystemTime=1689227989937 2023-07-13 05:59:49,973 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd. 2023-07-13 05:59:49,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd. 2023-07-13 05:59:49,974 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=c96fda412d307d645bd66b267f4787dd, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:49,974 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1689227989974"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689227989974"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689227989974"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689227989974"}]},"ts":"1689227989974"} 2023-07-13 05:59:49,981 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-07-13 05:59:49,981 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; OpenRegionProcedure c96fda412d307d645bd66b267f4787dd, server=jenkins-hbase20.apache.org,35187,1689227983701 in 193 msec 2023-07-13 05:59:49,984 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=11 2023-07-13 05:59:49,985 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=11, state=SUCCESS; TransitRegionStateProcedure table=testRematerializedTablesDoNoInheritSpace0, region=c96fda412d307d645bd66b267f4787dd, ASSIGN in 355 msec 2023-07-13 05:59:49,986 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=11, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRematerializedTablesDoNoInheritSpace0 execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-13 05:59:49,986 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRematerializedTablesDoNoInheritSpace0","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689227989986"}]},"ts":"1689227989986"} 2023-07-13 05:59:49,988 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testRematerializedTablesDoNoInheritSpace0, state=ENABLED in hbase:meta 2023-07-13 05:59:49,990 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=11, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRematerializedTablesDoNoInheritSpace0 execute state=CREATE_TABLE_POST_OPERATION 2023-07-13 05:59:49,993 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=11, state=SUCCESS; CreateTableProcedure table=testRematerializedTablesDoNoInheritSpace0 in 835 msec 2023-07-13 05:59:50,277 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x1a74b541 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 05:59:50,283 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1540fb32, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 05:59:50,287 DEBUG [hconnection-0x30cc2ea8-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 05:59:50,290 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:49932, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 05:59:50,290 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1230): Checking to see if procedure is done pid=11 2023-07-13 05:59:50,291 INFO [Listener at localhost.localdomain/45717] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testRematerializedTablesDoNoInheritSpace0, procId: 11 completed 2023-07-13 05:59:50,291 INFO [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots(360): Writing data 2023-07-13 05:59:50,370 DEBUG [master/jenkins-hbase20:0.Chore.1] quotas.QuotaObserverChore(336): testRematerializedTablesDoNoInheritSpace0 moved into observance of table space quota. 2023-07-13 05:59:50,379 DEBUG [regionserver/jenkins-hbase20:0.Chore.1] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-07-13 05:59:50,380 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:36042, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=MasterService 2023-07-13 05:59:50,416 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1a74b541 to 127.0.0.1:55963 2023-07-13 05:59:50,416 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 05:59:50,985 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests(539): Data was written to HBase 2023-07-13 05:59:50,998 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-13 05:59:51,005 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] procedure.ProcedureCoordinator(165): Submitting procedure testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:51,005 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRematerializedTablesDoNoInheritSpace0' 2023-07-13 05:59:51,006 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-13 05:59:51,006 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRematerializedTablesDoNoInheritSpace0' starting 'acquire' 2023-07-13 05:59:51,006 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRematerializedTablesDoNoInheritSpace0', kicking off acquire phase on members. 2023-07-13 05:59:51,007 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:51,007 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:51,008 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-13 05:59:51,008 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRematerializedTablesDoNoInheritSpace0/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:51,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-13 05:59:51,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-13 05:59:51,009 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRematerializedTablesDoNoInheritSpace0/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:51,009 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-13 05:59:51,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:51,010 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:51,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-13 05:59:51,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:51,011 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:51,013 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:51,014 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRematerializedTablesDoNoInheritSpace0' with timeout 60000ms 2023-07-13 05:59:51,014 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-13 05:59:51,014 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRematerializedTablesDoNoInheritSpace0' starting 'acquire' stage 2023-07-13 05:59:51,015 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-13 05:59:51,015 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-13 05:59:51,016 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd. 2023-07-13 05:59:51,016 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd. started... 2023-07-13 05:59:51,017 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c96fda412d307d645bd66b267f4787dd 1/1 column families, dataSize=2.22 MB heapSize=2.81 MB 2023-07-13 05:59:51,232 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-13 05:59:51,233 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-13 05:59:51,234 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRematerializedTablesDoNoInheritSpace0' 2023-07-13 05:59:51,235 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:quota' 2023-07-13 05:59:51,304 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x1510f4d7 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 05:59:51,309 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3867cbb8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 05:59:51,312 DEBUG [hconnection-0x64a2053d-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 05:59:51,314 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:49942, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 05:59:51,542 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.22 MB at sequenceid=165 (bloomFilter=true), to=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testRematerializedTablesDoNoInheritSpace0/c96fda412d307d645bd66b267f4787dd/.tmp/f1/e341259944284fcbb3a271f7166fa651 2023-07-13 05:59:51,588 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testRematerializedTablesDoNoInheritSpace0/c96fda412d307d645bd66b267f4787dd/.tmp/f1/e341259944284fcbb3a271f7166fa651 as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testRematerializedTablesDoNoInheritSpace0/c96fda412d307d645bd66b267f4787dd/f1/e341259944284fcbb3a271f7166fa651 2023-07-13 05:59:51,600 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testRematerializedTablesDoNoInheritSpace0/c96fda412d307d645bd66b267f4787dd/f1/e341259944284fcbb3a271f7166fa651, entries=8192, sequenceid=165, filesize=2.3 M 2023-07-13 05:59:51,603 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.22 MB/2325418, heapSize ~2.81 MB/2949360, currentSize=0 B/0 for c96fda412d307d645bd66b267f4787dd in 586ms, sequenceid=165, compaction requested=false 2023-07-13 05:59:51,605 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c96fda412d307d645bd66b267f4787dd: 2023-07-13 05:59:51,605 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd. 2023-07-13 05:59:51,605 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-13 05:59:51,606 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-13 05:59:51,606 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:51,606 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRematerializedTablesDoNoInheritSpace0' locally acquired 2023-07-13 05:59:51,606 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,35187,1689227983701' joining acquired barrier for procedure (testRematerializedTablesDoNoInheritSpace0) in zk 2023-07-13 05:59:51,608 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRematerializedTablesDoNoInheritSpace0/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:51,608 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:51,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRematerializedTablesDoNoInheritSpace0/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:51,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 05:59:51,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 05:59:51,609 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:51,609 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRematerializedTablesDoNoInheritSpace0' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-13 05:59:51,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 05:59:51,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 05:59:51,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:51,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:51,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 05:59:51,611 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,35187,1689227983701' joining acquired barrier for procedure 'testRematerializedTablesDoNoInheritSpace0' on coordinator 2023-07-13 05:59:51,611 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRematerializedTablesDoNoInheritSpace0' starting 'in-barrier' execution. 2023-07-13 05:59:51,611 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@472d98bd[Count = 0] remaining members to acquire global barrier 2023-07-13 05:59:51,611 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:51,612 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:51,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:51,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:51,613 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:51,613 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRematerializedTablesDoNoInheritSpace0' received 'reached' from coordinator. 2023-07-13 05:59:51,613 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-13 05:59:51,613 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRematerializedTablesDoNoInheritSpace0' locally completed 2023-07-13 05:59:51,613 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRematerializedTablesDoNoInheritSpace0' completed for member 'jenkins-hbase20.apache.org,35187,1689227983701' in zk 2023-07-13 05:59:51,615 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:51,615 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRematerializedTablesDoNoInheritSpace0' has notified controller of completion 2023-07-13 05:59:51,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:51,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 05:59:51,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 05:59:51,615 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-13 05:59:51,615 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRematerializedTablesDoNoInheritSpace0' completed. 2023-07-13 05:59:51,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 05:59:51,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 05:59:51,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:51,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:51,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 05:59:51,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:51,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:51,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRematerializedTablesDoNoInheritSpace0' member 'jenkins-hbase20.apache.org,35187,1689227983701': 2023-07-13 05:59:51,618 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRematerializedTablesDoNoInheritSpace0' execution completed 2023-07-13 05:59:51,618 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,35187,1689227983701' released barrier for procedure'testRematerializedTablesDoNoInheritSpace0', counting down latch. Waiting for 0 more 2023-07-13 05:59:51,618 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-13 05:59:51,618 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-13 05:59:51,618 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:51,618 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRematerializedTablesDoNoInheritSpace0including nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-13 05:59:51,630 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:51,630 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-13 05:59:51,630 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:51,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:51,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:51,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:51,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 05:59:51,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 05:59:51,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-13 05:59:51,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-13 05:59:51,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 05:59:51,631 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRematerializedTablesDoNoInheritSpace0/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:51,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:51,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:51,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 05:59:51,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:51,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:51,633 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:51,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 05:59:51,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:51,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:51,641 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRematerializedTablesDoNoInheritSpace0/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:51,641 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-13 05:59:51,641 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:51,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-13 05:59:51,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-13 05:59:51,641 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-13 05:59:51,641 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRematerializedTablesDoNoInheritSpace0' 2023-07-13 05:59:51,642 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-13 05:59:51,641 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-13 05:59:51,641 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:51,642 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:51,642 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:51,643 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:51,643 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:51,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-13 05:59:51,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-13 05:59:51,645 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRematerializedTablesDoNoInheritSpace0'' to complete. (max 20000 ms per retry) 2023-07-13 05:59:51,645 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-13 05:59:51,746 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-13 05:59:51,750 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-13 05:59:51,751 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests(542): Data flushed to disk 2023-07-13 05:59:51,751 INFO [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots(372): Waiting until table size reflects written data 2023-07-13 05:59:51,752 INFO [Listener at localhost.localdomain/45717] hbase.Waiter(180): Waiting up to [30,000] milli-secs(wait.for.ratio=[1]) 2023-07-13 05:59:51,761 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for testRematerializedTablesDoNoInheritSpace0: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=0 B/1 GB] 2023-07-13 05:59:51,961 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-13 05:59:52,264 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for testRematerializedTablesDoNoInheritSpace0: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=0 B/1 GB] 2023-07-13 05:59:52,299 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x01db4c59 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 05:59:52,305 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@49814007, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 05:59:52,311 DEBUG [hconnection-0x29d5efd9-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 05:59:52,314 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:49950, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 05:59:52,765 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for testRematerializedTablesDoNoInheritSpace0: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=0 B/1 GB] 2023-07-13 05:59:52,901 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: Master,sub=Coprocessor.Master.CP_org.apache.hadoop.hbase.quotas.MasterQuotasObserver 2023-07-13 05:59:52,901 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering Master,sub=Coprocessor.Master.CP_org.apache.hadoop.hbase.quotas.MasterQuotasObserver Metrics about HBase MasterObservers 2023-07-13 05:59:52,902 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-13 05:59:52,902 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-07-13 05:59:53,226 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x18aef5fd to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 05:59:53,230 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@386d5752, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 05:59:53,232 DEBUG [hconnection-0x45f9eaf3-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 05:59:53,233 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:49960, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 05:59:53,258 DEBUG [master/jenkins-hbase20:0.Chore.1] quotas.QuotaObserverChore(336): testRematerializedTablesDoNoInheritSpace0 moved into observance of table space quota. 2023-07-13 05:59:53,267 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for testRematerializedTablesDoNoInheritSpace0: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=2.25 MB/1 GB] 2023-07-13 05:59:53,268 INFO [Listener at localhost.localdomain/45717] hbase.Waiter(180): Waiting up to [15,000] milli-secs(wait.for.ratio=[1]) 2023-07-13 05:59:53,269 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for testRematerializedTablesDoNoInheritSpace0: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=2.25 MB/1 GB] 2023-07-13 05:59:53,269 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$14(433): Last observed size=-1 2023-07-13 05:59:53,771 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for testRematerializedTablesDoNoInheritSpace0: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=2.25 MB/1 GB] 2023-07-13 05:59:53,772 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$14(433): Last observed size=2359562 2023-07-13 05:59:54,274 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for testRematerializedTablesDoNoInheritSpace0: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=2.25 MB/1 GB] 2023-07-13 05:59:54,274 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$14(433): Last observed size=2359562 2023-07-13 05:59:54,307 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x77565b57 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 05:59:54,312 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@336aeac9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 05:59:54,314 DEBUG [hconnection-0x75cd221f-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 05:59:54,316 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:49970, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 05:59:54,776 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for testRematerializedTablesDoNoInheritSpace0: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=2.25 MB/1 GB] 2023-07-13 05:59:54,776 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$14(433): Last observed size=2359562 2023-07-13 05:59:55,279 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for testRematerializedTablesDoNoInheritSpace0: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=2.25 MB/1 GB] 2023-07-13 05:59:55,279 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$14(433): Last observed size=2359562 2023-07-13 05:59:55,302 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x46d6892e to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 05:59:55,307 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1bb7b24e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 05:59:55,310 DEBUG [hconnection-0x595270f5-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 05:59:55,348 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:49986, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 05:59:55,781 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for testRematerializedTablesDoNoInheritSpace0: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=2.25 MB/1 GB] 2023-07-13 05:59:55,781 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$14(433): Last observed size=2359562 2023-07-13 05:59:55,783 INFO [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots(386): Initial table size was 2359562 2023-07-13 05:59:55,783 INFO [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots(388): Snapshot the table 2023-07-13 05:59:55,793 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1583): Client=jenkins//148.251.75.209 snapshot request for:{ ss=testRematerializedTablesDoNoInheritSpace0_snapshot1 table=testRematerializedTablesDoNoInheritSpace0 type=FLUSH ttl=0 } 2023-07-13 05:59:55,793 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1689227995793 (current time:1689227995793). 2023-07-13 05:59:55,794 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-07-13 05:59:55,794 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] zookeeper.ReadOnlyZKClient(139): Connect 0x0845ca4b to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 05:59:55,798 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@64ec4f2c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 05:59:55,802 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 05:59:55,803 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:49988, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 05:59:55,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0845ca4b to 127.0.0.1:55963 2023-07-13 05:59:55,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 05:59:55,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-07-13 05:59:55,806 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotManager(648): Table enabled, starting distributed snapshots for { ss=testRematerializedTablesDoNoInheritSpace0_snapshot1 table=testRematerializedTablesDoNoInheritSpace0 type=FLUSH ttl=0 } 2023-07-13 05:59:55,824 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] procedure2.ProcedureExecutor(1029): Stored pid=15, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testRematerializedTablesDoNoInheritSpace0, type=EXCLUSIVE 2023-07-13 05:59:55,824 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=15, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testRematerializedTablesDoNoInheritSpace0, type=EXCLUSIVE 2023-07-13 05:59:55,825 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=15, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testRematerializedTablesDoNoInheritSpace0, type=EXCLUSIVE; timeout=600000, timestamp=1689228595825 2023-07-13 05:59:55,825 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotManager(653): Started snapshot: { ss=testRematerializedTablesDoNoInheritSpace0_snapshot1 table=testRematerializedTablesDoNoInheritSpace0 type=FLUSH ttl=0 } 2023-07-13 05:59:55,826 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.TakeSnapshotHandler(174): Running FLUSH table snapshot testRematerializedTablesDoNoInheritSpace0_snapshot1 C_M_SNAPSHOT_TABLE on table testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:55,827 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=15, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testRematerializedTablesDoNoInheritSpace0, type=EXCLUSIVE 2023-07-13 05:59:55,828 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] procedure2.ProcedureExecutor(1029): Stored pid=16, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testRematerializedTablesDoNoInheritSpace0, type=SHARED 2023-07-13 05:59:55,828 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=15, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testRematerializedTablesDoNoInheritSpace0, type=EXCLUSIVE in 9 msec 2023-07-13 05:59:55,829 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=16, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testRematerializedTablesDoNoInheritSpace0, type=SHARED 2023-07-13 05:59:55,830 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=16, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testRematerializedTablesDoNoInheritSpace0, type=SHARED; timeout=600000, timestamp=1689228595830 2023-07-13 05:59:55,831 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=testRematerializedTablesDoNoInheritSpace0_snapshot1 table=testRematerializedTablesDoNoInheritSpace0 type=FLUSH ttl=0 }'' to complete. (max 20000 ms per retry) 2023-07-13 05:59:55,831 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-07-13 05:59:55,932 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-07-13 05:59:55,936 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=testRematerializedTablesDoNoInheritSpace0_snapshot1 table=testRematerializedTablesDoNoInheritSpace0 type=FLUSH ttl=0 } is done 2023-07-13 05:59:55,939 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotManager(404): Snapshoting '{ ss=testRematerializedTablesDoNoInheritSpace0_snapshot1 table=testRematerializedTablesDoNoInheritSpace0 type=FLUSH ttl=0 }' is still in progress! 2023-07-13 05:59:55,940 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-07-13 05:59:56,140 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-07-13 05:59:56,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=testRematerializedTablesDoNoInheritSpace0_snapshot1 table=testRematerializedTablesDoNoInheritSpace0 type=FLUSH ttl=0 } is done 2023-07-13 05:59:56,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotManager(404): Snapshoting '{ ss=testRematerializedTablesDoNoInheritSpace0_snapshot1 table=testRematerializedTablesDoNoInheritSpace0 type=FLUSH ttl=0 }' is still in progress! 2023-07-13 05:59:56,143 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-07-13 05:59:56,232 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x36181bf3 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 05:59:56,237 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7d990ad1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 05:59:56,239 DEBUG [hconnection-0x1f3cfb9b-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 05:59:56,241 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:50000, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 05:59:56,246 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] procedure.ProcedureCoordinator(165): Submitting procedure testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,246 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRematerializedTablesDoNoInheritSpace0_snapshot1' 2023-07-13 05:59:56,246 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-07-13 05:59:56,246 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRematerializedTablesDoNoInheritSpace0_snapshot1' starting 'acquire' 2023-07-13 05:59:56,246 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRematerializedTablesDoNoInheritSpace0_snapshot1', kicking off acquire phase on members. 2023-07-13 05:59:56,247 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,247 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/online-snapshot/acquired/testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,249 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/online-snapshot/acquired/testRematerializedTablesDoNoInheritSpace0_snapshot1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:56,249 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-07-13 05:59:56,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-07-13 05:59:56,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-13 05:59:56,249 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/acquired/testRematerializedTablesDoNoInheritSpace0_snapshot1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:56,249 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-13 05:59:56,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,250 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 113 2023-07-13 05:59:56,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,251 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot testRematerializedTablesDoNoInheritSpace0_snapshot1 from table testRematerializedTablesDoNoInheritSpace0 type FLUSH 2023-07-13 05:59:56,252 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,252 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRematerializedTablesDoNoInheritSpace0_snapshot1' with timeout 300000ms 2023-07-13 05:59:56,252 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-07-13 05:59:56,253 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRematerializedTablesDoNoInheritSpace0_snapshot1' starting 'acquire' stage 2023-07-13 05:59:56,253 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRematerializedTablesDoNoInheritSpace0_snapshot1' locally acquired 2023-07-13 05:59:56,253 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,35187,1689227983701' joining acquired barrier for procedure (testRematerializedTablesDoNoInheritSpace0_snapshot1) in zk 2023-07-13 05:59:56,254 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/acquired/testRematerializedTablesDoNoInheritSpace0_snapshot1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:56,254 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/acquired/testRematerializedTablesDoNoInheritSpace0_snapshot1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:56,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 05:59:56,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-07-13 05:59:56,255 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,255 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRematerializedTablesDoNoInheritSpace0_snapshot1' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-13 05:59:56,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 05:59:56,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 05:59:56,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:56,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 05:59:56,256 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,35187,1689227983701' joining acquired barrier for procedure 'testRematerializedTablesDoNoInheritSpace0_snapshot1' on coordinator 2023-07-13 05:59:56,256 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3c434ec9[Count = 0] remaining members to acquire global barrier 2023-07-13 05:59:56,256 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRematerializedTablesDoNoInheritSpace0_snapshot1' starting 'in-barrier' execution. 2023-07-13 05:59:56,257 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/online-snapshot/reached/testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,258 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,258 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRematerializedTablesDoNoInheritSpace0_snapshot1' received 'reached' from coordinator. 2023-07-13 05:59:56,258 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/testRematerializedTablesDoNoInheritSpace0_snapshot1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:56,258 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-13 05:59:56,259 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] snapshot.FlushSnapshotSubprocedure(170): Flush Snapshot Tasks submitted for 1 regions 2023-07-13 05:59:56,259 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd. 2023-07-13 05:59:56,259 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(301): Waiting for local region snapshots to finish. 2023-07-13 05:59:56,260 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd. started... 2023-07-13 05:59:56,260 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] regionserver.HRegion(2446): Flush status journal for c96fda412d307d645bd66b267f4787dd: 2023-07-13 05:59:56,261 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd.' region-info for snapshot=testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,267 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-07-13 05:59:56,272 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testRematerializedTablesDoNoInheritSpace0/c96fda412d307d645bd66b267f4787dd/f1/e341259944284fcbb3a271f7166fa651] hfiles 2023-07-13 05:59:56,272 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/1): hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testRematerializedTablesDoNoInheritSpace0/c96fda412d307d645bd66b267f4787dd/f1/e341259944284fcbb3a271f7166fa651 for snapshot=testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,293 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd. completed. 2023-07-13 05:59:56,293 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd. 2023-07-13 05:59:56,293 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 1/1 local region snapshots. 2023-07-13 05:59:56,293 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(314): Completed 1 local region snapshots. 2023-07-13 05:59:56,294 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(345): cancelling 0 tasks for snapshot jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:56,294 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRematerializedTablesDoNoInheritSpace0_snapshot1' locally completed 2023-07-13 05:59:56,294 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRematerializedTablesDoNoInheritSpace0_snapshot1' completed for member 'jenkins-hbase20.apache.org,35187,1689227983701' in zk 2023-07-13 05:59:56,297 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRematerializedTablesDoNoInheritSpace0_snapshot1' has notified controller of completion 2023-07-13 05:59:56,297 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-13 05:59:56,297 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/testRematerializedTablesDoNoInheritSpace0_snapshot1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:56,297 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRematerializedTablesDoNoInheritSpace0_snapshot1' completed. 2023-07-13 05:59:56,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/reached/testRematerializedTablesDoNoInheritSpace0_snapshot1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:56,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 05:59:56,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-07-13 05:59:56,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 05:59:56,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 05:59:56,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:56,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 05:59:56,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:56,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRematerializedTablesDoNoInheritSpace0_snapshot1' member 'jenkins-hbase20.apache.org,35187,1689227983701': 2023-07-13 05:59:56,300 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,35187,1689227983701' released barrier for procedure'testRematerializedTablesDoNoInheritSpace0_snapshot1', counting down latch. Waiting for 0 more 2023-07-13 05:59:56,300 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRematerializedTablesDoNoInheritSpace0_snapshot1' execution completed 2023-07-13 05:59:56,301 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-13 05:59:56,301 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-13 05:59:56,301 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,301 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRematerializedTablesDoNoInheritSpace0_snapshot1including nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2023-07-13 05:59:56,304 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,304 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/abort/testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 05:59:56,305 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-07-13 05:59:56,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-07-13 05:59:56,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-07-13 05:59:56,305 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/testRematerializedTablesDoNoInheritSpace0_snapshot1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:56,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-13 05:59:56,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 05:59:56,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 05:59:56,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,307 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/testRematerializedTablesDoNoInheritSpace0_snapshot1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:56,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:56,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 05:59:56,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:56,313 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/testRematerializedTablesDoNoInheritSpace0_snapshot1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:56,313 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-07-13 05:59:56,313 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired/testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-07-13 05:59:56,313 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.EnabledTableSnapshotHandler(97): Done waiting - online snapshot for testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,313 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,313 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-07-13 05:59:56,313 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,313 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-13 05:59:56,313 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Unable to get data of znode /hbase/online-snapshot/abort/testRematerializedTablesDoNoInheritSpace0_snapshot1 because node does not exist (not an error) 2023-07-13 05:59:56,313 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/testRematerializedTablesDoNoInheritSpace0_snapshot1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:56,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-13 05:59:56,314 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/reached/testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,314 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,314 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/abort/testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,314 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-07-13 05:59:56,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-13 05:59:56,316 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.hbase-snapshot/.tmp/testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,443 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-07-13 05:59:56,445 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=testRematerializedTablesDoNoInheritSpace0_snapshot1 table=testRematerializedTablesDoNoInheritSpace0 type=FLUSH ttl=0 } is done 2023-07-13 05:59:56,445 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotManager(404): Snapshoting '{ ss=testRematerializedTablesDoNoInheritSpace0_snapshot1 table=testRematerializedTablesDoNoInheritSpace0 type=FLUSH ttl=0 }' is still in progress! 2023-07-13 05:59:56,445 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-07-13 05:59:56,762 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.hbase-snapshot/.tmp/testRematerializedTablesDoNoInheritSpace0_snapshot1 to hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.hbase-snapshot/testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:56,945 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-07-13 05:59:56,947 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=testRematerializedTablesDoNoInheritSpace0_snapshot1 table=testRematerializedTablesDoNoInheritSpace0 type=FLUSH ttl=0 } is done 2023-07-13 05:59:56,947 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotManager(404): Snapshoting '{ ss=testRematerializedTablesDoNoInheritSpace0_snapshot1 table=testRematerializedTablesDoNoInheritSpace0 type=FLUSH ttl=0 }' is still in progress! 2023-07-13 05:59:56,947 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-07-13 05:59:57,225 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x2afd11bb to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 05:59:57,230 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@381bdf6d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 05:59:57,232 DEBUG [hconnection-0x1da5bfcf-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 05:59:57,235 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:50012, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 05:59:57,602 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.TakeSnapshotHandler(229): Snapshot testRematerializedTablesDoNoInheritSpace0_snapshot1 of table testRematerializedTablesDoNoInheritSpace0 completed 2023-07-13 05:59:57,603 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.hbase-snapshot/.tmp/testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:57,606 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.hbase-snapshot/.tmp/testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:57,606 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running FLUSH table snapshot testRematerializedTablesDoNoInheritSpace0_snapshot1 C_M_SNAPSHOT_TABLE on table testRematerializedTablesDoNoInheritSpace0 at 1689227995826Consolidate snapshot: testRematerializedTablesDoNoInheritSpace0_snapshot1 at 1689227996313 (+487 ms)Loading Region manifests for testRematerializedTablesDoNoInheritSpace0_snapshot1 at 1689227996314 (+1 ms)Writing data manifest for testRematerializedTablesDoNoInheritSpace0_snapshot1 at 1689227996324 (+10 ms)Verifying snapshot: testRematerializedTablesDoNoInheritSpace0_snapshot1 at 1689227996740 (+416 ms)Snapshot testRematerializedTablesDoNoInheritSpace0_snapshot1 of table testRematerializedTablesDoNoInheritSpace0 completed at 1689227997602 (+862 ms) 2023-07-13 05:59:57,607 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=16, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testRematerializedTablesDoNoInheritSpace0, type=SHARED 2023-07-13 05:59:57,610 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=16, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testRematerializedTablesDoNoInheritSpace0, type=SHARED in 1.7810 sec 2023-07-13 05:59:57,947 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-07-13 05:59:57,951 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=testRematerializedTablesDoNoInheritSpace0_snapshot1 table=testRematerializedTablesDoNoInheritSpace0 type=FLUSH ttl=0 } is done 2023-07-13 05:59:57,952 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotManager(401): Snapshot '{ ss=testRematerializedTablesDoNoInheritSpace0_snapshot1 table=testRematerializedTablesDoNoInheritSpace0 type=FLUSH ttl=0 }' has completed, notifying client. 2023-07-13 05:59:57,985 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] procedure2.ProcedureExecutor(1029): Stored pid=17, state=RUNNABLE:CLONE_SNAPSHOT_PRE_OPERATION; CloneSnapshotProcedure (table=testRematerializedTablesDoNoInheritSpace1 snapshot=name: "testRematerializedTablesDoNoInheritSpace0_snapshot1" table: "testRematerializedTablesDoNoInheritSpace0" creation_time: 1689227995793 type: FLUSH version: 2 ttl: 0 ) 2023-07-13 05:59:57,985 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotManager(750): Clone snapshot=testRematerializedTablesDoNoInheritSpace0_snapshot1 as table=testRematerializedTablesDoNoInheritSpace1 2023-07-13 05:59:57,988 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1230): Checking to see if procedure is done pid=17 2023-07-13 05:59:58,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1230): Checking to see if procedure is done pid=17 2023-07-13 05:59:58,226 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x436b75bd to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 05:59:58,235 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@221733c7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 05:59:58,240 DEBUG [hconnection-0x1cda8129-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 05:59:58,243 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:50026, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 05:59:58,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1230): Checking to see if procedure is done pid=17 2023-07-13 05:59:58,406 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testRematerializedTablesDoNoInheritSpace1/.tabledesc/.tableinfo.0000000001 2023-07-13 05:59:58,411 INFO [PEWorker-5] snapshot.RestoreSnapshotHelper(177): starting restore table regions using snapshot=name: "testRematerializedTablesDoNoInheritSpace0_snapshot1" table: "testRematerializedTablesDoNoInheritSpace0" creation_time: 1689227995793 type: FLUSH version: 2 ttl: 0 2023-07-13 05:59:58,411 DEBUG [PEWorker-5] snapshot.RestoreSnapshotHelper(785): get table regions: hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testRematerializedTablesDoNoInheritSpace1 2023-07-13 05:59:58,412 INFO [PEWorker-5] snapshot.RestoreSnapshotHelper(239): region to add: c96fda412d307d645bd66b267f4787dd 2023-07-13 05:59:58,412 INFO [PEWorker-5] snapshot.RestoreSnapshotHelper(585): clone region=c96fda412d307d645bd66b267f4787dd as 777f666dd9b789f8aac944c77f179207 in snapshot testRematerializedTablesDoNoInheritSpace0_snapshot1 2023-07-13 05:59:58,414 INFO [RestoreSnapshot-pool-0] regionserver.HRegion(7675): creating {ENCODED => 777f666dd9b789f8aac944c77f179207, NAME => 'testRematerializedTablesDoNoInheritSpace1,,1689227989151.777f666dd9b789f8aac944c77f179207.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRematerializedTablesDoNoInheritSpace1', {NAME => 'f1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp 2023-07-13 05:59:58,425 DEBUG [RestoreSnapshot-pool-0] regionserver.HRegion(866): Instantiated testRematerializedTablesDoNoInheritSpace1,,1689227989151.777f666dd9b789f8aac944c77f179207.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 05:59:58,425 INFO [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(628): Adding HFileLink e341259944284fcbb3a271f7166fa651 from cloned region in snapshot testRematerializedTablesDoNoInheritSpace0_snapshot1 to table=testRematerializedTablesDoNoInheritSpace1 2023-07-13 05:59:58,432 DEBUG [RestoreSnapshot-pool-0] regionserver.HRegion(1604): Closing 777f666dd9b789f8aac944c77f179207, disabling compactions & flushes 2023-07-13 05:59:58,432 INFO [RestoreSnapshot-pool-0] regionserver.HRegion(1626): Closing region testRematerializedTablesDoNoInheritSpace1,,1689227989151.777f666dd9b789f8aac944c77f179207. 2023-07-13 05:59:58,432 DEBUG [RestoreSnapshot-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRematerializedTablesDoNoInheritSpace1,,1689227989151.777f666dd9b789f8aac944c77f179207. 2023-07-13 05:59:58,432 DEBUG [RestoreSnapshot-pool-0] regionserver.HRegion(1714): Acquired close lock on testRematerializedTablesDoNoInheritSpace1,,1689227989151.777f666dd9b789f8aac944c77f179207. after waiting 0 ms 2023-07-13 05:59:58,432 DEBUG [RestoreSnapshot-pool-0] regionserver.HRegion(1724): Updates disabled for region testRematerializedTablesDoNoInheritSpace1,,1689227989151.777f666dd9b789f8aac944c77f179207. 2023-07-13 05:59:58,432 INFO [RestoreSnapshot-pool-0] regionserver.HRegion(1838): Closed testRematerializedTablesDoNoInheritSpace1,,1689227989151.777f666dd9b789f8aac944c77f179207. 2023-07-13 05:59:58,432 DEBUG [RestoreSnapshot-pool-0] regionserver.HRegion(1558): Region close journal for 777f666dd9b789f8aac944c77f179207: 2023-07-13 05:59:58,433 INFO [PEWorker-5] snapshot.RestoreSnapshotHelper(266): finishing restore table regions using snapshot=name: "testRematerializedTablesDoNoInheritSpace0_snapshot1" table: "testRematerializedTablesDoNoInheritSpace0" creation_time: 1689227995793 type: FLUSH version: 2 ttl: 0 2023-07-13 05:59:58,433 INFO [PEWorker-5] procedure.CloneSnapshotProcedure$1(421): Clone snapshot=testRematerializedTablesDoNoInheritSpace0_snapshot1 on table=testRematerializedTablesDoNoInheritSpace1 completed! 2023-07-13 05:59:58,438 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRematerializedTablesDoNoInheritSpace1,,1689227989151.777f666dd9b789f8aac944c77f179207.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1689227998438"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689227998438"}]},"ts":"1689227998438"} 2023-07-13 05:59:58,440 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-13 05:59:58,441 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRematerializedTablesDoNoInheritSpace1","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689227998441"}]},"ts":"1689227998441"} 2023-07-13 05:59:58,443 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testRematerializedTablesDoNoInheritSpace1, state=ENABLING in hbase:meta 2023-07-13 05:59:58,447 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRematerializedTablesDoNoInheritSpace1, region=777f666dd9b789f8aac944c77f179207, ASSIGN}] 2023-07-13 05:59:58,449 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=18, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRematerializedTablesDoNoInheritSpace1, region=777f666dd9b789f8aac944c77f179207, ASSIGN 2023-07-13 05:59:58,450 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=18, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRematerializedTablesDoNoInheritSpace1, region=777f666dd9b789f8aac944c77f179207, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,35187,1689227983701; forceNewPlan=false, retain=false 2023-07-13 05:59:58,597 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1230): Checking to see if procedure is done pid=17 2023-07-13 05:59:58,601 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=18 updating hbase:meta row=777f666dd9b789f8aac944c77f179207, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:58,602 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRematerializedTablesDoNoInheritSpace1,,1689227989151.777f666dd9b789f8aac944c77f179207.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1689227998601"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689227998601"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689227998601"}]},"ts":"1689227998601"} 2023-07-13 05:59:58,607 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=19, ppid=18, state=RUNNABLE; OpenRegionProcedure 777f666dd9b789f8aac944c77f179207, server=jenkins-hbase20.apache.org,35187,1689227983701}] 2023-07-13 05:59:58,772 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testRematerializedTablesDoNoInheritSpace1,,1689227989151.777f666dd9b789f8aac944c77f179207. 2023-07-13 05:59:58,772 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 777f666dd9b789f8aac944c77f179207, NAME => 'testRematerializedTablesDoNoInheritSpace1,,1689227989151.777f666dd9b789f8aac944c77f179207.', STARTKEY => '', ENDKEY => ''} 2023-07-13 05:59:58,773 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRematerializedTablesDoNoInheritSpace1 777f666dd9b789f8aac944c77f179207 2023-07-13 05:59:58,773 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testRematerializedTablesDoNoInheritSpace1,,1689227989151.777f666dd9b789f8aac944c77f179207.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 05:59:58,773 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 777f666dd9b789f8aac944c77f179207 2023-07-13 05:59:58,773 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 777f666dd9b789f8aac944c77f179207 2023-07-13 05:59:58,777 INFO [StoreOpener-777f666dd9b789f8aac944c77f179207-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region 777f666dd9b789f8aac944c77f179207 2023-07-13 05:59:58,779 DEBUG [StoreOpener-777f666dd9b789f8aac944c77f179207-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testRematerializedTablesDoNoInheritSpace1/777f666dd9b789f8aac944c77f179207/f1 2023-07-13 05:59:58,779 DEBUG [StoreOpener-777f666dd9b789f8aac944c77f179207-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testRematerializedTablesDoNoInheritSpace1/777f666dd9b789f8aac944c77f179207/f1 2023-07-13 05:59:58,779 INFO [StoreOpener-777f666dd9b789f8aac944c77f179207-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 777f666dd9b789f8aac944c77f179207 columnFamilyName f1 2023-07-13 05:59:58,790 DEBUG [StoreOpener-777f666dd9b789f8aac944c77f179207-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testRematerializedTablesDoNoInheritSpace1/777f666dd9b789f8aac944c77f179207/f1/testRematerializedTablesDoNoInheritSpace0=c96fda412d307d645bd66b267f4787dd-e341259944284fcbb3a271f7166fa651 2023-07-13 05:59:58,791 INFO [StoreOpener-777f666dd9b789f8aac944c77f179207-1] regionserver.HStore(310): Store=777f666dd9b789f8aac944c77f179207/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-13 05:59:58,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testRematerializedTablesDoNoInheritSpace1/777f666dd9b789f8aac944c77f179207 2023-07-13 05:59:58,793 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testRematerializedTablesDoNoInheritSpace1/777f666dd9b789f8aac944c77f179207 2023-07-13 05:59:58,796 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 777f666dd9b789f8aac944c77f179207 2023-07-13 05:59:58,799 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testRematerializedTablesDoNoInheritSpace1/777f666dd9b789f8aac944c77f179207/recovered.edits/165.seqid, newMaxSeqId=165, maxSeqId=-1 2023-07-13 05:59:58,799 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 777f666dd9b789f8aac944c77f179207; next sequenceid=166; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63564312, jitterRate=-0.0528179407119751}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-13 05:59:58,800 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 777f666dd9b789f8aac944c77f179207: 2023-07-13 05:59:58,801 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRematerializedTablesDoNoInheritSpace1,,1689227989151.777f666dd9b789f8aac944c77f179207., pid=19, masterSystemTime=1689227998762 2023-07-13 05:59:58,802 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRematerializedTablesDoNoInheritSpace1,,1689227989151.777f666dd9b789f8aac944c77f179207. 2023-07-13 05:59:58,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testRematerializedTablesDoNoInheritSpace1,,1689227989151.777f666dd9b789f8aac944c77f179207. 2023-07-13 05:59:58,803 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=18 updating hbase:meta row=777f666dd9b789f8aac944c77f179207, regionState=OPEN, openSeqNum=166, regionLocation=jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:58,803 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRematerializedTablesDoNoInheritSpace1,,1689227989151.777f666dd9b789f8aac944c77f179207.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1689227998803"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689227998803"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689227998803"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689227998803"}]},"ts":"1689227998803"} 2023-07-13 05:59:58,807 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=19, resume processing ppid=18 2023-07-13 05:59:58,807 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=19, ppid=18, state=SUCCESS; OpenRegionProcedure 777f666dd9b789f8aac944c77f179207, server=jenkins-hbase20.apache.org,35187,1689227983701 in 198 msec 2023-07-13 05:59:58,809 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=17 2023-07-13 05:59:58,809 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=17, state=SUCCESS; TransitRegionStateProcedure table=testRematerializedTablesDoNoInheritSpace1, region=777f666dd9b789f8aac944c77f179207, ASSIGN in 360 msec 2023-07-13 05:59:58,810 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRematerializedTablesDoNoInheritSpace1","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689227998810"}]},"ts":"1689227998810"} 2023-07-13 05:59:58,812 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testRematerializedTablesDoNoInheritSpace1, state=ENABLED in hbase:meta 2023-07-13 05:59:58,817 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=17, state=SUCCESS; CloneSnapshotProcedure (table=testRematerializedTablesDoNoInheritSpace1 snapshot=name: "testRematerializedTablesDoNoInheritSpace0_snapshot1" table: "testRematerializedTablesDoNoInheritSpace0" creation_time: 1689227995793 type: FLUSH version: 2 ttl: 0 ) in 839 msec 2023-07-13 05:59:59,099 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1230): Checking to see if procedure is done pid=17 2023-07-13 05:59:59,099 INFO [Listener at localhost.localdomain/45717] client.HBaseAdmin$TableFuture(3541): Operation: MODIFY, Table Name: default:testRematerializedTablesDoNoInheritSpace1, procId: 17 completed 2023-07-13 05:59:59,231 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x6e62e24c to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 05:59:59,241 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1cd7df8a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 05:59:59,248 DEBUG [hconnection-0x73b60fcc-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 05:59:59,250 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:42068, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 05:59:59,321 DEBUG [master/jenkins-hbase20:0.Chore.1] quotas.QuotaObserverChore(336): testRematerializedTablesDoNoInheritSpace1 moved into observance of table space quota. 2023-07-13 05:59:59,598 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests(539): Data was written to HBase 2023-07-13 05:59:59,603 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-13 05:59:59,610 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] procedure.ProcedureCoordinator(143): Procedure testRematerializedTablesDoNoInheritSpace0 was in running list but was completed. Accepting new attempt. 2023-07-13 05:59:59,611 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] procedure.ProcedureCoordinator(165): Submitting procedure testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:59,611 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRematerializedTablesDoNoInheritSpace0' 2023-07-13 05:59:59,611 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-13 05:59:59,611 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRematerializedTablesDoNoInheritSpace0' starting 'acquire' 2023-07-13 05:59:59,612 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRematerializedTablesDoNoInheritSpace0', kicking off acquire phase on members. 2023-07-13 05:59:59,612 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:59,612 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:59,614 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-13 05:59:59,614 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-13 05:59:59,614 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRematerializedTablesDoNoInheritSpace0/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:59,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-13 05:59:59,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:59,615 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:59,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-13 05:59:59,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:59,615 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:59,616 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRematerializedTablesDoNoInheritSpace0/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 05:59:59,616 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-13 05:59:59,618 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRematerializedTablesDoNoInheritSpace0 is still present, removing 2023-07-13 05:59:59,619 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRematerializedTablesDoNoInheritSpace0 2023-07-13 05:59:59,619 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRematerializedTablesDoNoInheritSpace0' with timeout 60000ms 2023-07-13 05:59:59,619 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-13 05:59:59,620 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRematerializedTablesDoNoInheritSpace0' starting 'acquire' stage 2023-07-13 05:59:59,623 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-13 05:59:59,623 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-13 05:59:59,630 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd. 2023-07-13 05:59:59,631 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd. started... 2023-07-13 05:59:59,631 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing c96fda412d307d645bd66b267f4787dd 1/1 column families, dataSize=2.22 MB heapSize=2.81 MB 2023-07-13 06:00:00,107 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.22 MB at sequenceid=330 (bloomFilter=true), to=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testRematerializedTablesDoNoInheritSpace0/c96fda412d307d645bd66b267f4787dd/.tmp/f1/40de9141a9df439f9c469ab4df51c3df 2023-07-13 06:00:00,121 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testRematerializedTablesDoNoInheritSpace0/c96fda412d307d645bd66b267f4787dd/.tmp/f1/40de9141a9df439f9c469ab4df51c3df as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testRematerializedTablesDoNoInheritSpace0/c96fda412d307d645bd66b267f4787dd/f1/40de9141a9df439f9c469ab4df51c3df 2023-07-13 06:00:00,131 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testRematerializedTablesDoNoInheritSpace0/c96fda412d307d645bd66b267f4787dd/f1/40de9141a9df439f9c469ab4df51c3df, entries=8192, sequenceid=330, filesize=2.3 M 2023-07-13 06:00:00,133 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.22 MB/2325418, heapSize ~2.81 MB/2949360, currentSize=0 B/0 for c96fda412d307d645bd66b267f4787dd in 502ms, sequenceid=330, compaction requested=false 2023-07-13 06:00:00,133 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c96fda412d307d645bd66b267f4787dd: 2023-07-13 06:00:00,134 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd. 2023-07-13 06:00:00,134 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-13 06:00:00,134 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-13 06:00:00,134 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:00,135 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRematerializedTablesDoNoInheritSpace0' locally acquired 2023-07-13 06:00:00,135 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,35187,1689227983701' joining acquired barrier for procedure (testRematerializedTablesDoNoInheritSpace0) in zk 2023-07-13 06:00:00,137 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,137 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRematerializedTablesDoNoInheritSpace0/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:00,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRematerializedTablesDoNoInheritSpace0/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:00,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:00,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:00,137 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,137 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRematerializedTablesDoNoInheritSpace0' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-13 06:00:00,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:00,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:00,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:00,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:00,139 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,35187,1689227983701' joining acquired barrier for procedure 'testRematerializedTablesDoNoInheritSpace0' on coordinator 2023-07-13 06:00:00,139 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3741cef9[Count = 0] remaining members to acquire global barrier 2023-07-13 06:00:00,139 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRematerializedTablesDoNoInheritSpace0' starting 'in-barrier' execution. 2023-07-13 06:00:00,139 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,140 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,140 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRematerializedTablesDoNoInheritSpace0' received 'reached' from coordinator. 2023-07-13 06:00:00,140 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRematerializedTablesDoNoInheritSpace0' locally completed 2023-07-13 06:00:00,140 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:00,140 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-13 06:00:00,140 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRematerializedTablesDoNoInheritSpace0' completed for member 'jenkins-hbase20.apache.org,35187,1689227983701' in zk 2023-07-13 06:00:00,142 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRematerializedTablesDoNoInheritSpace0' has notified controller of completion 2023-07-13 06:00:00,142 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:00,142 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-13 06:00:00,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:00,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:00,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:00,142 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRematerializedTablesDoNoInheritSpace0' completed. 2023-07-13 06:00:00,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:00,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:00,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:00,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:00,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:00,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRematerializedTablesDoNoInheritSpace0' member 'jenkins-hbase20.apache.org,35187,1689227983701': 2023-07-13 06:00:00,145 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,35187,1689227983701' released barrier for procedure'testRematerializedTablesDoNoInheritSpace0', counting down latch. Waiting for 0 more 2023-07-13 06:00:00,145 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRematerializedTablesDoNoInheritSpace0' execution completed 2023-07-13 06:00:00,145 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-13 06:00:00,145 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-13 06:00:00,145 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,146 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRematerializedTablesDoNoInheritSpace0including nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-13 06:00:00,154 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,154 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,154 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-13 06:00:00,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:00,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:00,155 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-13 06:00:00,155 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRematerializedTablesDoNoInheritSpace0/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:00,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-13 06:00:00,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:00,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:00,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,157 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:00,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:00,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:00,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:00,163 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-13 06:00:00,163 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRematerializedTablesDoNoInheritSpace0/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:00,163 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-13 06:00:00,163 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testRematerializedTablesDoNoInheritSpace0 because node does not exist (not an error) 2023-07-13 06:00:00,163 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,163 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRematerializedTablesDoNoInheritSpace0' 2023-07-13 06:00:00,164 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-13 06:00:00,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-13 06:00:00,163 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-13 06:00:00,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-13 06:00:00,164 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,164 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:00,164 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,164 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,164 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRematerializedTablesDoNoInheritSpace0'' to complete. (max 20000 ms per retry) 2023-07-13 06:00:00,164 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,164 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-13 06:00:00,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-13 06:00:00,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-13 06:00:00,264 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-13 06:00:00,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-13 06:00:00,267 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests(542): Data flushed to disk 2023-07-13 06:00:00,273 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-13 06:00:00,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] procedure.ProcedureCoordinator(143): Procedure testRematerializedTablesDoNoInheritSpace0 was in running list but was completed. Accepting new attempt. 2023-07-13 06:00:00,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] procedure.ProcedureCoordinator(165): Submitting procedure testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,277 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRematerializedTablesDoNoInheritSpace0' 2023-07-13 06:00:00,277 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-13 06:00:00,278 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRematerializedTablesDoNoInheritSpace0' starting 'acquire' 2023-07-13 06:00:00,278 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRematerializedTablesDoNoInheritSpace0', kicking off acquire phase on members. 2023-07-13 06:00:00,278 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,278 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,279 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-13 06:00:00,279 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRematerializedTablesDoNoInheritSpace0/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:00,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-13 06:00:00,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-13 06:00:00,280 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRematerializedTablesDoNoInheritSpace0/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:00,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,280 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-13 06:00:00,280 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-13 06:00:00,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,281 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,281 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRematerializedTablesDoNoInheritSpace0 is still present, removing 2023-07-13 06:00:00,281 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,281 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRematerializedTablesDoNoInheritSpace0' with timeout 60000ms 2023-07-13 06:00:00,281 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-13 06:00:00,282 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRematerializedTablesDoNoInheritSpace0' starting 'acquire' stage 2023-07-13 06:00:00,282 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-13 06:00:00,282 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-13 06:00:00,282 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd. 2023-07-13 06:00:00,282 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd. started... 2023-07-13 06:00:00,282 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for c96fda412d307d645bd66b267f4787dd: 2023-07-13 06:00:00,282 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd. 2023-07-13 06:00:00,282 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-13 06:00:00,282 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-13 06:00:00,282 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:00,282 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRematerializedTablesDoNoInheritSpace0' locally acquired 2023-07-13 06:00:00,282 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,35187,1689227983701' joining acquired barrier for procedure (testRematerializedTablesDoNoInheritSpace0) in zk 2023-07-13 06:00:00,284 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,284 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRematerializedTablesDoNoInheritSpace0/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:00,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRematerializedTablesDoNoInheritSpace0/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:00,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:00,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:00,285 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,285 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRematerializedTablesDoNoInheritSpace0' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-13 06:00:00,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:00,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:00,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:00,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:00,286 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,35187,1689227983701' joining acquired barrier for procedure 'testRematerializedTablesDoNoInheritSpace0' on coordinator 2023-07-13 06:00:00,286 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7e05e33c[Count = 0] remaining members to acquire global barrier 2023-07-13 06:00:00,286 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRematerializedTablesDoNoInheritSpace0' starting 'in-barrier' execution. 2023-07-13 06:00:00,287 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,288 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,289 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRematerializedTablesDoNoInheritSpace0' received 'reached' from coordinator. 2023-07-13 06:00:00,289 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRematerializedTablesDoNoInheritSpace0' locally completed 2023-07-13 06:00:00,289 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRematerializedTablesDoNoInheritSpace0' completed for member 'jenkins-hbase20.apache.org,35187,1689227983701' in zk 2023-07-13 06:00:00,289 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:00,289 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-13 06:00:00,291 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRematerializedTablesDoNoInheritSpace0' has notified controller of completion 2023-07-13 06:00:00,291 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:00,291 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-13 06:00:00,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:00,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:00,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:00,291 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRematerializedTablesDoNoInheritSpace0' completed. 2023-07-13 06:00:00,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:00,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:00,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:00,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:00,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:00,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRematerializedTablesDoNoInheritSpace0' member 'jenkins-hbase20.apache.org,35187,1689227983701': 2023-07-13 06:00:00,295 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,35187,1689227983701' released barrier for procedure'testRematerializedTablesDoNoInheritSpace0', counting down latch. Waiting for 0 more 2023-07-13 06:00:00,295 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRematerializedTablesDoNoInheritSpace0' execution completed 2023-07-13 06:00:00,295 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-13 06:00:00,295 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-13 06:00:00,295 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,295 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRematerializedTablesDoNoInheritSpace0including nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-13 06:00:00,301 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x2ae92c5a to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:00,304 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,304 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,304 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:00,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:00,304 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-13 06:00:00,305 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-13 06:00:00,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-13 06:00:00,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:00,305 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRematerializedTablesDoNoInheritSpace0/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:00,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,306 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@65cc8dee, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:00,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:00,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:00,307 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:00,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:00,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,308 DEBUG [hconnection-0x71bde78e-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:00,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:00,310 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:42076, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:00,311 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRematerializedTablesDoNoInheritSpace0/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:00,311 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,311 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-13 06:00:00,311 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-13 06:00:00,311 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-13 06:00:00,311 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,312 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:00,312 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,312 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,312 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRematerializedTablesDoNoInheritSpace0 2023-07-13 06:00:00,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-13 06:00:00,311 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-13 06:00:00,311 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRematerializedTablesDoNoInheritSpace0' 2023-07-13 06:00:00,312 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-13 06:00:00,313 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRematerializedTablesDoNoInheritSpace0'' to complete. (max 20000 ms per retry) 2023-07-13 06:00:00,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-13 06:00:00,313 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-13 06:00:00,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-13 06:00:00,413 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-13 06:00:00,415 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-13 06:00:00,416 INFO [Listener at localhost.localdomain/45717] hbase.Waiter(180): Waiting up to [30,000] milli-secs(wait.for.ratio=[1]) 2023-07-13 06:00:00,418 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for testRematerializedTablesDoNoInheritSpace0: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=2.25 MB/1 GB] 2023-07-13 06:00:01,273 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x2a7a8e51 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:01,278 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6ecaf73f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:01,280 DEBUG [hconnection-0x4e462d4-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:01,281 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:42090, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:01,329 DEBUG [master/jenkins-hbase20:0.Chore.1] quotas.QuotaObserverChore(336): testRematerializedTablesDoNoInheritSpace0 moved into observance of table space quota. 2023-07-13 06:00:01,420 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for testRematerializedTablesDoNoInheritSpace0: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=4.51 MB/1 GB] 2023-07-13 06:00:01,423 DEBUG [Listener at localhost.localdomain/45717] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-07-13 06:00:01,424 DEBUG [Listener at localhost.localdomain/45717] regionserver.HStore(1912): 777f666dd9b789f8aac944c77f179207/f1 is initiating major compaction (all files) 2023-07-13 06:00:01,424 INFO [Listener at localhost.localdomain/45717] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-13 06:00:01,424 INFO [Listener at localhost.localdomain/45717] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-13 06:00:01,425 INFO [Listener at localhost.localdomain/45717] regionserver.HRegion(2259): Starting compaction of 777f666dd9b789f8aac944c77f179207/f1 in testRematerializedTablesDoNoInheritSpace1,,1689227989151.777f666dd9b789f8aac944c77f179207. 2023-07-13 06:00:01,425 INFO [Listener at localhost.localdomain/45717] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testRematerializedTablesDoNoInheritSpace1/777f666dd9b789f8aac944c77f179207/f1/testRematerializedTablesDoNoInheritSpace0=c96fda412d307d645bd66b267f4787dd-e341259944284fcbb3a271f7166fa651] into tmpdir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testRematerializedTablesDoNoInheritSpace1/777f666dd9b789f8aac944c77f179207/.tmp, totalSize=2.3 M 2023-07-13 06:00:01,426 DEBUG [Listener at localhost.localdomain/45717] compactions.Compactor(207): Compacting testRematerializedTablesDoNoInheritSpace0=c96fda412d307d645bd66b267f4787dd-e341259944284fcbb3a271f7166fa651, keycount=8192, bloomtype=ROW, size=2.3 M, encoding=NONE, compression=NONE, seqNum=165, earliestPutTs=1689227990384 2023-07-13 06:00:01,548 INFO [Listener at localhost.localdomain/45717] throttle.PressureAwareThroughputController(145): 777f666dd9b789f8aac944c77f179207#f1#compaction#2 average throughput is 21.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-13 06:00:01,564 DEBUG [Listener at localhost.localdomain/45717] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testRematerializedTablesDoNoInheritSpace1/777f666dd9b789f8aac944c77f179207/.tmp/f1/79f760c2ca2947b0987243888b7b5abc as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testRematerializedTablesDoNoInheritSpace1/777f666dd9b789f8aac944c77f179207/f1/79f760c2ca2947b0987243888b7b5abc 2023-07-13 06:00:01,582 DEBUG [Listener at localhost.localdomain/45717] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRematerializedTablesDoNoInheritSpace1' 2023-07-13 06:00:01,583 INFO [Listener at localhost.localdomain/45717] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 777f666dd9b789f8aac944c77f179207/f1 of 777f666dd9b789f8aac944c77f179207 into 79f760c2ca2947b0987243888b7b5abc(size=2.3 M), total size for store is 2.3 M. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-13 06:00:01,583 DEBUG [Listener at localhost.localdomain/45717] regionserver.HRegion(2289): Compaction status journal for 777f666dd9b789f8aac944c77f179207: 2023-07-13 06:00:01,584 INFO [Listener at localhost.localdomain/45717] hbase.Waiter(180): Waiting up to [30,000] milli-secs(wait.for.ratio=[1]) 2023-07-13 06:00:01,586 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for testRematerializedTablesDoNoInheritSpace1: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=0 B/1 GB] 2023-07-13 06:00:02,225 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x6239991b to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:02,230 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@38de647a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:02,232 DEBUG [hconnection-0x35a70ec7-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:02,234 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:42104, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:02,588 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for testRematerializedTablesDoNoInheritSpace1: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=0 B/1 GB] 2023-07-13 06:00:03,241 DEBUG [master/jenkins-hbase20:0.Chore.1] quotas.QuotaObserverChore(336): testRematerializedTablesDoNoInheritSpace1 moved into observance of table space quota. 2023-07-13 06:00:03,241 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x2a039616 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:03,247 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6c79743a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:03,249 DEBUG [hconnection-0x5faf088a-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:03,253 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:42116, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:03,589 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for testRematerializedTablesDoNoInheritSpace1: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=2.25 MB/1 GB] 2023-07-13 06:00:03,606 INFO [Listener at localhost.localdomain/45717] hbase.ResourceChecker(175): after: quotas.TestSpaceQuotasWithSnapshots#testRematerializedTablesDoNoInheritSpace Thread=271 (was 211) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x2a7a8e51-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x436b75bd-SendThread(127.0.0.1:55963) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x35a70ec7-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x2afd11bb-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x1510f4d7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$46/955501514.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x77565b57-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x01db4c59 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$46/955501514.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x77565b57 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$46/955501514.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x2a7a8e51-SendThread(127.0.0.1:55963) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: hconnection-0x5faf088a-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x6e62e24c sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$46/955501514.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x6e62e24c-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x36181bf3-SendThread(127.0.0.1:55963) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x36181bf3-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-216011253_17 at /127.0.0.1:42580 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x18aef5fd sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$46/955501514.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x56316acd-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x55e0d78d-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x1510f4d7-SendThread(127.0.0.1:55963) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x6239991b-SendThread(127.0.0.1:55963) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x2ae92c5a-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: hconnection-0x5faf088a-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x18aef5fd-SendThread(127.0.0.1:55963) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x77565b57-SendThread(127.0.0.1:55963) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x1510f4d7-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-216011253_17 at /127.0.0.1:39808 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x01db4c59-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: hconnection-0x56316acd-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x2ae92c5a sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$46/955501514.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x2ae92c5a-SendThread(127.0.0.1:55963) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x18aef5fd-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x436b75bd sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$46/955501514.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4e462d4-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/74b62a1d-1af8-4c02-70de-8b0007eeebf7/cluster_3a5fcc1f-2335-da3f-c526-d77046317035/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x46d6892e-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x436b75bd-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: hconnection-0x55e0d78d-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x2a7a8e51 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$46/955501514.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x6e62e24c-SendThread(127.0.0.1:55963) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x2a039616-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: hconnection-0x4e462d4-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/74b62a1d-1af8-4c02-70de-8b0007eeebf7/cluster_3a5fcc1f-2335-da3f-c526-d77046317035/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: hconnection-0x55e0d78d-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x2afd11bb-SendThread(127.0.0.1:55963) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x6239991b-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x2a039616-SendThread(127.0.0.1:55963) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: hconnection-0x56316acd-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x36181bf3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$46/955501514.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x6239991b sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$46/955501514.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x46d6892e-SendThread(127.0.0.1:55963) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: hconnection-0x35a70ec7-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x55e0d78d-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x46d6892e sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$46/955501514.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1754673348_17 at /127.0.0.1:39824 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x2a039616 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$46/955501514.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x2afd11bb sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$46/955501514.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:55963@0x01db4c59-SendThread(127.0.0.1:55963) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) - Thread LEAK? -, OpenFileDescriptor=580 (was 473) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=174 (was 178), ProcessCount=168 (was 165) - ProcessCount LEAK? -, AvailableMemoryMB=7100 (was 6911) - AvailableMemoryMB LEAK? - 2023-07-13 06:00:03,617 INFO [Listener at localhost.localdomain/45717] hbase.ResourceChecker(147): before: quotas.TestSpaceQuotasWithSnapshots#testNamespacesInheritSnapshotSize Thread=271, OpenFileDescriptor=580, MaxFileDescriptor=60000, SystemLoadAverage=174, ProcessCount=168, AvailableMemoryMB=7099 2023-07-13 06:00:03,621 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.HMaster$15(3014): Client=jenkins//148.251.75.209 creating {NAME => 'ns2'} 2023-07-13 06:00:03,623 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] procedure2.ProcedureExecutor(1029): Stored pid=20, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=ns2 2023-07-13 06:00:03,628 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1230): Checking to see if procedure is done pid=20 2023-07-13 06:00:03,630 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-07-13 06:00:03,633 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=20, state=SUCCESS; CreateNamespaceProcedure, namespace=ns2 in 10 msec 2023-07-13 06:00:03,730 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1230): Checking to see if procedure is done pid=20 2023-07-13 06:00:03,732 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.HMaster$4(2112): Client=jenkins//148.251.75.209 create 'ns2:testNamespacesInheritSnapshotSize3', {NAME => 'f1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-13 06:00:03,733 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] procedure2.ProcedureExecutor(1029): Stored pid=21, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:03,735 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=21, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=ns2:testNamespacesInheritSnapshotSize3 execute state=CREATE_TABLE_PRE_OPERATION 2023-07-13 06:00:03,735 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(700): Client=jenkins//148.251.75.209 procedure request for creating table: namespace: "ns2" qualifier: "testNamespacesInheritSnapshotSize3" procId is: 21 2023-07-13 06:00:03,736 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=21, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=ns2:testNamespacesInheritSnapshotSize3 execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-13 06:00:03,736 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1230): Checking to see if procedure is done pid=21 2023-07-13 06:00:03,737 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/ns2/testNamespacesInheritSnapshotSize3/b6e943f7ce0da690354636beb379d14f 2023-07-13 06:00:03,738 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/ns2/testNamespacesInheritSnapshotSize3/b6e943f7ce0da690354636beb379d14f empty. 2023-07-13 06:00:03,739 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/ns2/testNamespacesInheritSnapshotSize3/b6e943f7ce0da690354636beb379d14f 2023-07-13 06:00:03,739 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived ns2:testNamespacesInheritSnapshotSize3 regions 2023-07-13 06:00:03,838 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1230): Checking to see if procedure is done pid=21 2023-07-13 06:00:04,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1230): Checking to see if procedure is done pid=21 2023-07-13 06:00:04,152 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/ns2/testNamespacesInheritSnapshotSize3/.tabledesc/.tableinfo.0000000001 2023-07-13 06:00:04,154 INFO [RegionOpenAndInit-ns2:testNamespacesInheritSnapshotSize3-pool-0] regionserver.HRegion(7675): creating {ENCODED => b6e943f7ce0da690354636beb379d14f, NAME => 'ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='ns2:testNamespacesInheritSnapshotSize3', {NAME => 'f1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp 2023-07-13 06:00:04,225 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x0efd70d1 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:04,230 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@17616d44, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:04,232 DEBUG [hconnection-0x49ca0fdc-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:04,233 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:42118, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:04,309 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-13 06:00:04,340 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1230): Checking to see if procedure is done pid=21 2023-07-13 06:00:04,567 DEBUG [RegionOpenAndInit-ns2:testNamespacesInheritSnapshotSize3-pool-0] regionserver.HRegion(866): Instantiated ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 06:00:04,567 DEBUG [RegionOpenAndInit-ns2:testNamespacesInheritSnapshotSize3-pool-0] regionserver.HRegion(1604): Closing b6e943f7ce0da690354636beb379d14f, disabling compactions & flushes 2023-07-13 06:00:04,567 INFO [RegionOpenAndInit-ns2:testNamespacesInheritSnapshotSize3-pool-0] regionserver.HRegion(1626): Closing region ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f. 2023-07-13 06:00:04,567 DEBUG [RegionOpenAndInit-ns2:testNamespacesInheritSnapshotSize3-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f. 2023-07-13 06:00:04,567 DEBUG [RegionOpenAndInit-ns2:testNamespacesInheritSnapshotSize3-pool-0] regionserver.HRegion(1714): Acquired close lock on ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f. after waiting 0 ms 2023-07-13 06:00:04,567 DEBUG [RegionOpenAndInit-ns2:testNamespacesInheritSnapshotSize3-pool-0] regionserver.HRegion(1724): Updates disabled for region ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f. 2023-07-13 06:00:04,567 INFO [RegionOpenAndInit-ns2:testNamespacesInheritSnapshotSize3-pool-0] regionserver.HRegion(1838): Closed ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f. 2023-07-13 06:00:04,567 DEBUG [RegionOpenAndInit-ns2:testNamespacesInheritSnapshotSize3-pool-0] regionserver.HRegion(1558): Region close journal for b6e943f7ce0da690354636beb379d14f: 2023-07-13 06:00:04,570 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=21, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=ns2:testNamespacesInheritSnapshotSize3 execute state=CREATE_TABLE_ADD_TO_META 2023-07-13 06:00:04,571 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689228004571"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689228004571"}]},"ts":"1689228004571"} 2023-07-13 06:00:04,572 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-13 06:00:04,573 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=21, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=ns2:testNamespacesInheritSnapshotSize3 execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-13 06:00:04,573 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns2:testNamespacesInheritSnapshotSize3","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689228004573"}]},"ts":"1689228004573"} 2023-07-13 06:00:04,575 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=ns2:testNamespacesInheritSnapshotSize3, state=ENABLING in hbase:meta 2023-07-13 06:00:04,577 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns2:testNamespacesInheritSnapshotSize3, region=b6e943f7ce0da690354636beb379d14f, ASSIGN}] 2023-07-13 06:00:04,579 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=22, ppid=21, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns2:testNamespacesInheritSnapshotSize3, region=b6e943f7ce0da690354636beb379d14f, ASSIGN 2023-07-13 06:00:04,580 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=22, ppid=21, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns2:testNamespacesInheritSnapshotSize3, region=b6e943f7ce0da690354636beb379d14f, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,35187,1689227983701; forceNewPlan=false, retain=false 2023-07-13 06:00:04,731 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=b6e943f7ce0da690354636beb379d14f, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:04,731 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689228004731"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689228004731"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689228004731"}]},"ts":"1689228004731"} 2023-07-13 06:00:04,733 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=22, state=RUNNABLE; OpenRegionProcedure b6e943f7ce0da690354636beb379d14f, server=jenkins-hbase20.apache.org,35187,1689227983701}] 2023-07-13 06:00:04,842 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1230): Checking to see if procedure is done pid=21 2023-07-13 06:00:04,889 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f. 2023-07-13 06:00:04,889 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b6e943f7ce0da690354636beb379d14f, NAME => 'ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f.', STARTKEY => '', ENDKEY => ''} 2023-07-13 06:00:04,889 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testNamespacesInheritSnapshotSize3 b6e943f7ce0da690354636beb379d14f 2023-07-13 06:00:04,890 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 06:00:04,890 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for b6e943f7ce0da690354636beb379d14f 2023-07-13 06:00:04,890 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for b6e943f7ce0da690354636beb379d14f 2023-07-13 06:00:04,891 INFO [StoreOpener-b6e943f7ce0da690354636beb379d14f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region b6e943f7ce0da690354636beb379d14f 2023-07-13 06:00:04,893 DEBUG [StoreOpener-b6e943f7ce0da690354636beb379d14f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/ns2/testNamespacesInheritSnapshotSize3/b6e943f7ce0da690354636beb379d14f/f1 2023-07-13 06:00:04,893 DEBUG [StoreOpener-b6e943f7ce0da690354636beb379d14f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/ns2/testNamespacesInheritSnapshotSize3/b6e943f7ce0da690354636beb379d14f/f1 2023-07-13 06:00:04,894 INFO [StoreOpener-b6e943f7ce0da690354636beb379d14f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b6e943f7ce0da690354636beb379d14f columnFamilyName f1 2023-07-13 06:00:04,894 INFO [StoreOpener-b6e943f7ce0da690354636beb379d14f-1] regionserver.HStore(310): Store=b6e943f7ce0da690354636beb379d14f/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-13 06:00:04,895 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/ns2/testNamespacesInheritSnapshotSize3/b6e943f7ce0da690354636beb379d14f 2023-07-13 06:00:04,896 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/ns2/testNamespacesInheritSnapshotSize3/b6e943f7ce0da690354636beb379d14f 2023-07-13 06:00:04,900 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for b6e943f7ce0da690354636beb379d14f 2023-07-13 06:00:04,903 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/ns2/testNamespacesInheritSnapshotSize3/b6e943f7ce0da690354636beb379d14f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-13 06:00:04,903 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened b6e943f7ce0da690354636beb379d14f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73423873, jitterRate=0.0941009670495987}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-13 06:00:04,903 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for b6e943f7ce0da690354636beb379d14f: 2023-07-13 06:00:04,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f., pid=23, masterSystemTime=1689228004884 2023-07-13 06:00:04,906 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f. 2023-07-13 06:00:04,906 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f. 2023-07-13 06:00:04,907 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=22 updating hbase:meta row=b6e943f7ce0da690354636beb379d14f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:04,907 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689228004907"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689228004907"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689228004907"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689228004907"}]},"ts":"1689228004907"} 2023-07-13 06:00:04,911 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=22 2023-07-13 06:00:04,911 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=22, state=SUCCESS; OpenRegionProcedure b6e943f7ce0da690354636beb379d14f, server=jenkins-hbase20.apache.org,35187,1689227983701 in 176 msec 2023-07-13 06:00:04,913 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-07-13 06:00:04,914 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; TransitRegionStateProcedure table=ns2:testNamespacesInheritSnapshotSize3, region=b6e943f7ce0da690354636beb379d14f, ASSIGN in 334 msec 2023-07-13 06:00:04,915 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=21, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=ns2:testNamespacesInheritSnapshotSize3 execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-13 06:00:04,915 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns2:testNamespacesInheritSnapshotSize3","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689228004915"}]},"ts":"1689228004915"} 2023-07-13 06:00:04,916 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=ns2:testNamespacesInheritSnapshotSize3, state=ENABLED in hbase:meta 2023-07-13 06:00:04,918 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=21, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=ns2:testNamespacesInheritSnapshotSize3 execute state=CREATE_TABLE_POST_OPERATION 2023-07-13 06:00:04,920 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=21, state=SUCCESS; CreateTableProcedure table=ns2:testNamespacesInheritSnapshotSize3 in 1.1860 sec 2023-07-13 06:00:05,235 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x713d5c74 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:05,243 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@57f552bf, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:05,247 DEBUG [hconnection-0x184e0b7c-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:05,249 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:42124, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:05,359 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testRematerializedTablesDoNoInheritSpace1/777f666dd9b789f8aac944c77f179207/f1/testRematerializedTablesDoNoInheritSpace0=c96fda412d307d645bd66b267f4787dd-e341259944284fcbb3a271f7166fa651] to archive 2023-07-13 06:00:05,361 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-07-13 06:00:05,368 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testRematerializedTablesDoNoInheritSpace1/777f666dd9b789f8aac944c77f179207/f1/testRematerializedTablesDoNoInheritSpace0=c96fda412d307d645bd66b267f4787dd-e341259944284fcbb3a271f7166fa651 to hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/archive/data/default/testRematerializedTablesDoNoInheritSpace1/777f666dd9b789f8aac944c77f179207/f1/testRematerializedTablesDoNoInheritSpace0=c96fda412d307d645bd66b267f4787dd-e341259944284fcbb3a271f7166fa651 2023-07-13 06:00:05,843 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1230): Checking to see if procedure is done pid=21 2023-07-13 06:00:05,843 INFO [Listener at localhost.localdomain/45717] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: ns2:testNamespacesInheritSnapshotSize3, procId: 21 completed 2023-07-13 06:00:05,844 INFO [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots(205): Writing data 2023-07-13 06:00:06,297 INFO [master/jenkins-hbase20:0.Chore.1] quotas.QuotaObserverChore(402): ns2:testNamespacesInheritSnapshotSize3 moving into observance of namespace space quota 2023-07-13 06:00:06,299 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x0297a855 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:06,319 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5b6c773, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:06,328 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests(539): Data was written to HBase 2023-07-13 06:00:06,333 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-13 06:00:06,347 DEBUG [hconnection-0x20eee246-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:06,363 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] procedure.ProcedureCoordinator(165): Submitting procedure ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,364 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns2:testNamespacesInheritSnapshotSize3' 2023-07-13 06:00:06,365 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-13 06:00:06,365 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns2:testNamespacesInheritSnapshotSize3' starting 'acquire' 2023-07-13 06:00:06,365 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:42134, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:06,365 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns2:testNamespacesInheritSnapshotSize3', kicking off acquire phase on members. 2023-07-13 06:00:06,366 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,366 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,368 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:06,368 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-13 06:00:06,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-13 06:00:06,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-13 06:00:06,373 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:06,373 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-13 06:00:06,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,375 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-13 06:00:06,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,376 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,402 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,403 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns2:testNamespacesInheritSnapshotSize3' with timeout 60000ms 2023-07-13 06:00:06,403 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-13 06:00:06,406 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns2:testNamespacesInheritSnapshotSize3' starting 'acquire' stage 2023-07-13 06:00:06,421 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-13 06:00:06,421 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-13 06:00:06,422 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f. 2023-07-13 06:00:06,422 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f. started... 2023-07-13 06:00:06,424 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b6e943f7ce0da690354636beb379d14f 1/1 column families, dataSize=2.22 MB heapSize=2.81 MB 2023-07-13 06:00:06,540 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.22 MB at sequenceid=165 (bloomFilter=true), to=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/ns2/testNamespacesInheritSnapshotSize3/b6e943f7ce0da690354636beb379d14f/.tmp/f1/30caf53d6cbf4d35bbd7ff6f186d8472 2023-07-13 06:00:06,554 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/ns2/testNamespacesInheritSnapshotSize3/b6e943f7ce0da690354636beb379d14f/.tmp/f1/30caf53d6cbf4d35bbd7ff6f186d8472 as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/ns2/testNamespacesInheritSnapshotSize3/b6e943f7ce0da690354636beb379d14f/f1/30caf53d6cbf4d35bbd7ff6f186d8472 2023-07-13 06:00:06,565 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/ns2/testNamespacesInheritSnapshotSize3/b6e943f7ce0da690354636beb379d14f/f1/30caf53d6cbf4d35bbd7ff6f186d8472, entries=8192, sequenceid=165, filesize=2.3 M 2023-07-13 06:00:06,567 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.22 MB/2325418, heapSize ~2.81 MB/2949360, currentSize=0 B/0 for b6e943f7ce0da690354636beb379d14f in 143ms, sequenceid=165, compaction requested=false 2023-07-13 06:00:06,567 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns2:testNamespacesInheritSnapshotSize3' 2023-07-13 06:00:06,568 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b6e943f7ce0da690354636beb379d14f: 2023-07-13 06:00:06,568 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f. 2023-07-13 06:00:06,568 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-13 06:00:06,569 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-13 06:00:06,569 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:06,569 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns2:testNamespacesInheritSnapshotSize3' locally acquired 2023-07-13 06:00:06,569 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,35187,1689227983701' joining acquired barrier for procedure (ns2:testNamespacesInheritSnapshotSize3) in zk 2023-07-13 06:00:06,570 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:06,570 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:06,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:06,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:06,571 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,571 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns2:testNamespacesInheritSnapshotSize3' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-13 06:00:06,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:06,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:06,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:06,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:06,573 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,35187,1689227983701' joining acquired barrier for procedure 'ns2:testNamespacesInheritSnapshotSize3' on coordinator 2023-07-13 06:00:06,573 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@55467431[Count = 0] remaining members to acquire global barrier 2023-07-13 06:00:06,573 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns2:testNamespacesInheritSnapshotSize3' starting 'in-barrier' execution. 2023-07-13 06:00:06,573 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,574 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,574 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,574 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns2:testNamespacesInheritSnapshotSize3' received 'reached' from coordinator. 2023-07-13 06:00:06,574 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns2:testNamespacesInheritSnapshotSize3' locally completed 2023-07-13 06:00:06,574 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns2:testNamespacesInheritSnapshotSize3' completed for member 'jenkins-hbase20.apache.org,35187,1689227983701' in zk 2023-07-13 06:00:06,575 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:06,575 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-13 06:00:06,576 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns2:testNamespacesInheritSnapshotSize3' has notified controller of completion 2023-07-13 06:00:06,576 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:06,576 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-13 06:00:06,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:06,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:06,576 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns2:testNamespacesInheritSnapshotSize3' completed. 2023-07-13 06:00:06,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:06,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:06,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:06,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:06,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:06,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:06,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns2:testNamespacesInheritSnapshotSize3' member 'jenkins-hbase20.apache.org,35187,1689227983701': 2023-07-13 06:00:06,580 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,35187,1689227983701' released barrier for procedure'ns2:testNamespacesInheritSnapshotSize3', counting down latch. Waiting for 0 more 2023-07-13 06:00:06,580 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns2:testNamespacesInheritSnapshotSize3' execution completed 2023-07-13 06:00:06,581 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-13 06:00:06,581 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-13 06:00:06,581 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,581 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns2:testNamespacesInheritSnapshotSize3including nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-13 06:00:06,588 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,588 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-13 06:00:06,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,588 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:06,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:06,589 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:06,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-13 06:00:06,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-13 06:00:06,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:06,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:06,592 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:06,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:06,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:06,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:06,596 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-13 06:00:06,596 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-13 06:00:06,596 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:06,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-13 06:00:06,596 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-13 06:00:06,596 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,596 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns2:testNamespacesInheritSnapshotSize3' 2023-07-13 06:00:06,597 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-13 06:00:06,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-13 06:00:06,597 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,597 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:06,597 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,597 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,597 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns2:testNamespacesInheritSnapshotSize3'' to complete. (max 20000 ms per retry) 2023-07-13 06:00:06,598 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 because node does not exist (not an error) 2023-07-13 06:00:06,597 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,598 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-13 06:00:06,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-13 06:00:06,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-13 06:00:06,698 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-13 06:00:06,699 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-13 06:00:06,699 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests(542): Data flushed to disk 2023-07-13 06:00:06,704 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-13 06:00:06,706 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] procedure.ProcedureCoordinator(143): Procedure ns2:testNamespacesInheritSnapshotSize3 was in running list but was completed. Accepting new attempt. 2023-07-13 06:00:06,706 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] procedure.ProcedureCoordinator(165): Submitting procedure ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,706 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns2:testNamespacesInheritSnapshotSize3' 2023-07-13 06:00:06,706 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-13 06:00:06,707 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns2:testNamespacesInheritSnapshotSize3' starting 'acquire' 2023-07-13 06:00:06,707 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns2:testNamespacesInheritSnapshotSize3', kicking off acquire phase on members. 2023-07-13 06:00:06,707 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,707 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,708 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:06,708 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-13 06:00:06,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-13 06:00:06,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-13 06:00:06,709 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:06,709 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-13 06:00:06,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,709 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-13 06:00:06,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,710 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,710 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns2:testNamespacesInheritSnapshotSize3 is still present, removing 2023-07-13 06:00:06,710 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,710 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns2:testNamespacesInheritSnapshotSize3' with timeout 60000ms 2023-07-13 06:00:06,710 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-13 06:00:06,711 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns2:testNamespacesInheritSnapshotSize3' starting 'acquire' stage 2023-07-13 06:00:06,711 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-13 06:00:06,711 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-13 06:00:06,711 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f. 2023-07-13 06:00:06,711 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f. started... 2023-07-13 06:00:06,711 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b6e943f7ce0da690354636beb379d14f: 2023-07-13 06:00:06,711 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f. 2023-07-13 06:00:06,711 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-13 06:00:06,711 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-13 06:00:06,712 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:06,712 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns2:testNamespacesInheritSnapshotSize3' locally acquired 2023-07-13 06:00:06,712 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,35187,1689227983701' joining acquired barrier for procedure (ns2:testNamespacesInheritSnapshotSize3) in zk 2023-07-13 06:00:06,713 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:06,713 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:06,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:06,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:06,714 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,714 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns2:testNamespacesInheritSnapshotSize3' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-13 06:00:06,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:06,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:06,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:06,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:06,716 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,35187,1689227983701' joining acquired barrier for procedure 'ns2:testNamespacesInheritSnapshotSize3' on coordinator 2023-07-13 06:00:06,716 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@73555d56[Count = 0] remaining members to acquire global barrier 2023-07-13 06:00:06,716 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns2:testNamespacesInheritSnapshotSize3' starting 'in-barrier' execution. 2023-07-13 06:00:06,716 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,718 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,719 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns2:testNamespacesInheritSnapshotSize3' received 'reached' from coordinator. 2023-07-13 06:00:06,719 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:06,719 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-13 06:00:06,719 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns2:testNamespacesInheritSnapshotSize3' locally completed 2023-07-13 06:00:06,719 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns2:testNamespacesInheritSnapshotSize3' completed for member 'jenkins-hbase20.apache.org,35187,1689227983701' in zk 2023-07-13 06:00:06,720 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:06,720 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns2:testNamespacesInheritSnapshotSize3' has notified controller of completion 2023-07-13 06:00:06,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:06,720 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-13 06:00:06,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:06,720 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns2:testNamespacesInheritSnapshotSize3' completed. 2023-07-13 06:00:06,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:06,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:06,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:06,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:06,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:06,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:06,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns2:testNamespacesInheritSnapshotSize3' member 'jenkins-hbase20.apache.org,35187,1689227983701': 2023-07-13 06:00:06,726 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,35187,1689227983701' released barrier for procedure'ns2:testNamespacesInheritSnapshotSize3', counting down latch. Waiting for 0 more 2023-07-13 06:00:06,726 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns2:testNamespacesInheritSnapshotSize3' execution completed 2023-07-13 06:00:06,726 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-13 06:00:06,726 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-13 06:00:06,726 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,726 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns2:testNamespacesInheritSnapshotSize3including nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-13 06:00:06,728 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,729 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,729 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:06,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:06,729 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-13 06:00:06,729 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:06,729 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-13 06:00:06,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-13 06:00:06,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:06,734 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:06,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:06,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:06,746 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-13 06:00:06,746 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-13 06:00:06,746 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:06,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-13 06:00:06,746 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,746 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-13 06:00:06,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-13 06:00:06,746 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns2:testNamespacesInheritSnapshotSize3' 2023-07-13 06:00:06,747 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-13 06:00:06,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:06,746 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,748 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:06,748 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,748 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,748 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:06,748 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns2:testNamespacesInheritSnapshotSize3'' to complete. (max 20000 ms per retry) 2023-07-13 06:00:06,748 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-13 06:00:06,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-13 06:00:06,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-13 06:00:06,749 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 because node does not exist (not an error) 2023-07-13 06:00:06,848 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-13 06:00:06,850 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-13 06:00:06,850 INFO [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots(216): Waiting until namespace size reflects written data 2023-07-13 06:00:06,851 INFO [Listener at localhost.localdomain/45717] hbase.Waiter(180): Waiting up to [30,000] milli-secs(wait.for.ratio=[1]) 2023-07-13 06:00:06,857 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for ns2: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=0 B/1 GB] 2023-07-13 06:00:07,287 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x21652020 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:07,292 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1f2e7661, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:07,295 DEBUG [hconnection-0x28671f32-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:07,297 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:42144, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:07,358 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for ns2: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=0 B/1 GB] 2023-07-13 06:00:07,860 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for ns2: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=0 B/1 GB] 2023-07-13 06:00:08,227 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x17ee0b90 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:08,233 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@17f8d71d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:08,237 DEBUG [hconnection-0x700bc007-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:08,240 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:42154, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:08,291 INFO [master/jenkins-hbase20:0.Chore.1] quotas.QuotaObserverChore(402): ns2:testNamespacesInheritSnapshotSize3 moving into observance of namespace space quota 2023-07-13 06:00:08,362 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for ns2: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=2.25 MB/1 GB] 2023-07-13 06:00:08,362 INFO [Listener at localhost.localdomain/45717] hbase.Waiter(180): Waiting up to [15,000] milli-secs(wait.for.ratio=[1]) 2023-07-13 06:00:08,364 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for ns2: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=2.25 MB/1 GB] 2023-07-13 06:00:08,364 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$14(433): Last observed size=-1 2023-07-13 06:00:08,868 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for ns2: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=2.25 MB/1 GB] 2023-07-13 06:00:08,868 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$14(433): Last observed size=2359562 2023-07-13 06:00:09,225 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x0f7ac721 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:09,234 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4cd76fb7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:09,237 DEBUG [hconnection-0x3d941355-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:09,240 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:37614, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:09,370 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for ns2: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=2.25 MB/1 GB] 2023-07-13 06:00:09,370 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$14(433): Last observed size=2359562 2023-07-13 06:00:09,873 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for ns2: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=2.25 MB/1 GB] 2023-07-13 06:00:09,873 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$14(433): Last observed size=2359562 2023-07-13 06:00:10,252 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x212f25d3 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:10,257 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6b087fb8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:10,259 DEBUG [hconnection-0x158437be-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:10,260 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:37630, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:10,375 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for ns2: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=2.25 MB/1 GB] 2023-07-13 06:00:10,375 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$14(433): Last observed size=2359562 2023-07-13 06:00:10,395 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-13 06:00:10,876 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for ns2: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=2.25 MB/1 GB] 2023-07-13 06:00:10,877 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$14(433): Last observed size=2359562 2023-07-13 06:00:10,878 INFO [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots(230): Initial table size was 2359562 2023-07-13 06:00:10,878 INFO [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots(232): Snapshot the table 2023-07-13 06:00:10,879 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1583): Client=jenkins//148.251.75.209 snapshot request for:{ ss=testNamespacesInheritSnapshotSize3_snapshot1 table=ns2:testNamespacesInheritSnapshotSize3 type=FLUSH ttl=0 } 2023-07-13 06:00:10,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1689228010879 (current time:1689228010879). 2023-07-13 06:00:10,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-07-13 06:00:10,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] zookeeper.ReadOnlyZKClient(139): Connect 0x74995595 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:10,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5827de6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:10,888 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:10,891 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:37640, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:10,892 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x74995595 to 127.0.0.1:55963 2023-07-13 06:00:10,892 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:10,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-07-13 06:00:10,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotManager(648): Table enabled, starting distributed snapshots for { ss=testNamespacesInheritSnapshotSize3_snapshot1 table=ns2:testNamespacesInheritSnapshotSize3 type=FLUSH ttl=0 } 2023-07-13 06:00:10,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=ns2:testNamespacesInheritSnapshotSize3, type=EXCLUSIVE 2023-07-13 06:00:10,898 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=24, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=ns2:testNamespacesInheritSnapshotSize3, type=EXCLUSIVE 2023-07-13 06:00:10,899 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=24, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=ns2:testNamespacesInheritSnapshotSize3, type=EXCLUSIVE; timeout=600000, timestamp=1689228610899 2023-07-13 06:00:10,899 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.TakeSnapshotHandler(174): Running FLUSH table snapshot testNamespacesInheritSnapshotSize3_snapshot1 C_M_SNAPSHOT_TABLE on table ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:10,899 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotManager(653): Started snapshot: { ss=testNamespacesInheritSnapshotSize3_snapshot1 table=ns2:testNamespacesInheritSnapshotSize3 type=FLUSH ttl=0 } 2023-07-13 06:00:10,901 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=testNamespacesInheritSnapshotSize3_snapshot1 table=ns2:testNamespacesInheritSnapshotSize3 type=FLUSH ttl=0 }'' to complete. (max 20000 ms per retry) 2023-07-13 06:00:10,901 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-07-13 06:00:10,901 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=24, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=ns2:testNamespacesInheritSnapshotSize3, type=EXCLUSIVE 2023-07-13 06:00:10,901 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=ns2:testNamespacesInheritSnapshotSize3, type=SHARED 2023-07-13 06:00:10,902 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=ns2:testNamespacesInheritSnapshotSize3, type=EXCLUSIVE in 5 msec 2023-07-13 06:00:10,903 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=25, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=ns2:testNamespacesInheritSnapshotSize3, type=SHARED 2023-07-13 06:00:10,903 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=25, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=ns2:testNamespacesInheritSnapshotSize3, type=SHARED; timeout=600000, timestamp=1689228610903 2023-07-13 06:00:11,001 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-07-13 06:00:11,003 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=testNamespacesInheritSnapshotSize3_snapshot1 table=ns2:testNamespacesInheritSnapshotSize3 type=FLUSH ttl=0 } is done 2023-07-13 06:00:11,003 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotManager(404): Snapshoting '{ ss=testNamespacesInheritSnapshotSize3_snapshot1 table=ns2:testNamespacesInheritSnapshotSize3 type=FLUSH ttl=0 }' is still in progress! 2023-07-13 06:00:11,003 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-07-13 06:00:11,203 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-07-13 06:00:11,205 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=testNamespacesInheritSnapshotSize3_snapshot1 table=ns2:testNamespacesInheritSnapshotSize3 type=FLUSH ttl=0 } is done 2023-07-13 06:00:11,205 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotManager(404): Snapshoting '{ ss=testNamespacesInheritSnapshotSize3_snapshot1 table=ns2:testNamespacesInheritSnapshotSize3 type=FLUSH ttl=0 }' is still in progress! 2023-07-13 06:00:11,205 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-07-13 06:00:11,312 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x3eacddbb to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:11,314 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] procedure.ProcedureCoordinator(165): Submitting procedure testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:11,314 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testNamespacesInheritSnapshotSize3_snapshot1' 2023-07-13 06:00:11,314 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-07-13 06:00:11,315 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testNamespacesInheritSnapshotSize3_snapshot1' starting 'acquire' 2023-07-13 06:00:11,315 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testNamespacesInheritSnapshotSize3_snapshot1', kicking off acquire phase on members. 2023-07-13 06:00:11,315 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:11,315 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/online-snapshot/acquired/testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:11,316 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2a1a7032, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:11,316 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-07-13 06:00:11,316 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/online-snapshot/acquired/testNamespacesInheritSnapshotSize3_snapshot1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:11,316 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-07-13 06:00:11,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-13 06:00:11,317 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/acquired/testNamespacesInheritSnapshotSize3_snapshot1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:11,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:11,317 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-13 06:00:11,317 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:11,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 103 2023-07-13 06:00:11,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:11,318 DEBUG [hconnection-0x159d23b9-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:11,318 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot testNamespacesInheritSnapshotSize3_snapshot1 from table ns2:testNamespacesInheritSnapshotSize3 type FLUSH 2023-07-13 06:00:11,318 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:11,318 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testNamespacesInheritSnapshotSize3_snapshot1' with timeout 300000ms 2023-07-13 06:00:11,318 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-07-13 06:00:11,319 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testNamespacesInheritSnapshotSize3_snapshot1' starting 'acquire' stage 2023-07-13 06:00:11,319 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testNamespacesInheritSnapshotSize3_snapshot1' locally acquired 2023-07-13 06:00:11,319 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,35187,1689227983701' joining acquired barrier for procedure (testNamespacesInheritSnapshotSize3_snapshot1) in zk 2023-07-13 06:00:11,320 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:37646, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:11,321 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/acquired/testNamespacesInheritSnapshotSize3_snapshot1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:11,321 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:11,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/acquired/testNamespacesInheritSnapshotSize3_snapshot1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:11,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:11,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-07-13 06:00:11,322 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:11,322 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testNamespacesInheritSnapshotSize3_snapshot1' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-13 06:00:11,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:11,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:11,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:11,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:11,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:11,325 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,35187,1689227983701' joining acquired barrier for procedure 'testNamespacesInheritSnapshotSize3_snapshot1' on coordinator 2023-07-13 06:00:11,325 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4a5a73c3[Count = 0] remaining members to acquire global barrier 2023-07-13 06:00:11,325 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testNamespacesInheritSnapshotSize3_snapshot1' starting 'in-barrier' execution. 2023-07-13 06:00:11,325 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/online-snapshot/reached/testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:11,326 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:11,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:11,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:11,326 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testNamespacesInheritSnapshotSize3_snapshot1' received 'reached' from coordinator. 2023-07-13 06:00:11,326 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/testNamespacesInheritSnapshotSize3_snapshot1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:11,326 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-13 06:00:11,326 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] snapshot.FlushSnapshotSubprocedure(170): Flush Snapshot Tasks submitted for 1 regions 2023-07-13 06:00:11,326 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f. 2023-07-13 06:00:11,326 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(301): Waiting for local region snapshots to finish. 2023-07-13 06:00:11,326 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f. started... 2023-07-13 06:00:11,326 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] regionserver.HRegion(2446): Flush status journal for b6e943f7ce0da690354636beb379d14f: 2023-07-13 06:00:11,327 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f.' region-info for snapshot=testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:11,327 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-07-13 06:00:11,327 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/ns2/testNamespacesInheritSnapshotSize3/b6e943f7ce0da690354636beb379d14f/f1/30caf53d6cbf4d35bbd7ff6f186d8472] hfiles 2023-07-13 06:00:11,327 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/1): hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/ns2/testNamespacesInheritSnapshotSize3/b6e943f7ce0da690354636beb379d14f/f1/30caf53d6cbf4d35bbd7ff6f186d8472 for snapshot=testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:11,506 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-07-13 06:00:11,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=testNamespacesInheritSnapshotSize3_snapshot1 table=ns2:testNamespacesInheritSnapshotSize3 type=FLUSH ttl=0 } is done 2023-07-13 06:00:11,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotManager(404): Snapshoting '{ ss=testNamespacesInheritSnapshotSize3_snapshot1 table=ns2:testNamespacesInheritSnapshotSize3 type=FLUSH ttl=0 }' is still in progress! 2023-07-13 06:00:11,507 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-07-13 06:00:11,738 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f. completed. 2023-07-13 06:00:11,738 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f. 2023-07-13 06:00:11,738 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 1/1 local region snapshots. 2023-07-13 06:00:11,738 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(314): Completed 1 local region snapshots. 2023-07-13 06:00:11,738 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(345): cancelling 0 tasks for snapshot jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:11,738 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testNamespacesInheritSnapshotSize3_snapshot1' locally completed 2023-07-13 06:00:11,738 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testNamespacesInheritSnapshotSize3_snapshot1' completed for member 'jenkins-hbase20.apache.org,35187,1689227983701' in zk 2023-07-13 06:00:11,741 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testNamespacesInheritSnapshotSize3_snapshot1' has notified controller of completion 2023-07-13 06:00:11,741 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/testNamespacesInheritSnapshotSize3_snapshot1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:11,741 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-13 06:00:11,741 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testNamespacesInheritSnapshotSize3_snapshot1' completed. 2023-07-13 06:00:11,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/reached/testNamespacesInheritSnapshotSize3_snapshot1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:11,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:11,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-07-13 06:00:11,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:11,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:11,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:11,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:11,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:11,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:11,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:11,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testNamespacesInheritSnapshotSize3_snapshot1' member 'jenkins-hbase20.apache.org,35187,1689227983701': 2023-07-13 06:00:11,768 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,35187,1689227983701' released barrier for procedure'testNamespacesInheritSnapshotSize3_snapshot1', counting down latch. Waiting for 0 more 2023-07-13 06:00:11,768 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testNamespacesInheritSnapshotSize3_snapshot1' execution completed 2023-07-13 06:00:11,768 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-13 06:00:11,768 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-13 06:00:11,768 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:11,768 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x21652020 to 127.0.0.1:55963 2023-07-13 06:00:11,768 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testNamespacesInheritSnapshotSize3_snapshot1including nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2023-07-13 06:00:11,768 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:11,768 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0297a855 to 127.0.0.1:55963 2023-07-13 06:00:11,769 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:11,769 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x17ee0b90 to 127.0.0.1:55963 2023-07-13 06:00:11,769 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:11,769 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0f7ac721 to 127.0.0.1:55963 2023-07-13 06:00:11,769 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:11,770 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x212f25d3 to 127.0.0.1:55963 2023-07-13 06:00:11,770 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:11,771 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:11,771 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3eacddbb to 127.0.0.1:55963 2023-07-13 06:00:11,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/abort/testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:11,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:11,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-07-13 06:00:11,771 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:11,771 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:11,771 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6239991b to 127.0.0.1:55963 2023-07-13 06:00:11,771 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-07-13 06:00:11,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:11,771 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:11,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:11,771 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2a039616 to 127.0.0.1:55963 2023-07-13 06:00:11,771 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:11,771 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0efd70d1 to 127.0.0.1:55963 2023-07-13 06:00:11,771 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:11,772 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x713d5c74 to 127.0.0.1:55963 2023-07-13 06:00:11,772 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:11,772 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x436b75bd to 127.0.0.1:55963 2023-07-13 06:00:11,772 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:11,772 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6e62e24c to 127.0.0.1:55963 2023-07-13 06:00:11,772 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:11,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-07-13 06:00:11,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-13 06:00:11,772 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2ae92c5a to 127.0.0.1:55963 2023-07-13 06:00:11,772 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:11,773 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2a7a8e51 to 127.0.0.1:55963 2023-07-13 06:00:11,773 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:11,773 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x36181bf3 to 127.0.0.1:55963 2023-07-13 06:00:11,773 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:11,773 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2afd11bb to 127.0.0.1:55963 2023-07-13 06:00:11,773 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:11,773 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x01db4c59 to 127.0.0.1:55963 2023-07-13 06:00:11,773 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:11,773 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x18aef5fd to 127.0.0.1:55963 2023-07-13 06:00:11,773 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:11,773 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x77565b57 to 127.0.0.1:55963 2023-07-13 06:00:11,773 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:11,774 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x46d6892e to 127.0.0.1:55963 2023-07-13 06:00:11,774 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:11,774 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1510f4d7 to 127.0.0.1:55963 2023-07-13 06:00:11,774 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:11,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:11,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:11,774 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/testNamespacesInheritSnapshotSize3_snapshot1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:11,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:11,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:11,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:11,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:11,777 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/testNamespacesInheritSnapshotSize3_snapshot1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:11,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:11,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:11,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:11,779 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/testNamespacesInheritSnapshotSize3_snapshot1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:11,779 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired/testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:11,779 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-07-13 06:00:11,779 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:11,779 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-13 06:00:11,779 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/testNamespacesInheritSnapshotSize3_snapshot1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:11,779 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.EnabledTableSnapshotHandler(97): Done waiting - online snapshot for testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:11,779 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-07-13 06:00:11,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-13 06:00:11,779 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-07-13 06:00:11,780 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:11,779 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/reached/testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:11,780 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:11,780 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/abort/testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:11,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-07-13 06:00:11,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-13 06:00:11,780 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.hbase-snapshot/.tmp/testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:12,007 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-07-13 06:00:12,010 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=testNamespacesInheritSnapshotSize3_snapshot1 table=ns2:testNamespacesInheritSnapshotSize3 type=FLUSH ttl=0 } is done 2023-07-13 06:00:12,010 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotManager(404): Snapshoting '{ ss=testNamespacesInheritSnapshotSize3_snapshot1 table=ns2:testNamespacesInheritSnapshotSize3 type=FLUSH ttl=0 }' is still in progress! 2023-07-13 06:00:12,011 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-07-13 06:00:12,207 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.hbase-snapshot/.tmp/testNamespacesInheritSnapshotSize3_snapshot1 to hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.hbase-snapshot/testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:12,259 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x4efa43a3 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:12,264 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@27706296, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:12,267 DEBUG [hconnection-0x474c99d9-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:12,269 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:37662, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:12,322 DEBUG [master/jenkins-hbase20:0.Chore.1] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.hbase-snapshot/testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:12,323 ERROR [master/jenkins-hbase20:0.Chore.1] hbase.ScheduledChore(166): Caught error java.lang.NullPointerException at org.apache.hadoop.hbase.quotas.FileArchiverNotifierImpl.getStoreFilesFromSnapshot(FileArchiverNotifierImpl.java:488) at org.apache.hadoop.hbase.quotas.FileArchiverNotifierImpl.computeSnapshotSizes(FileArchiverNotifierImpl.java:404) at org.apache.hadoop.hbase.quotas.FileArchiverNotifierImpl.computeAndStoreSnapshotSizes(FileArchiverNotifierImpl.java:334) at org.apache.hadoop.hbase.quotas.SnapshotQuotaObserverChore.computeSnapshotSizes(SnapshotQuotaObserverChore.java:225) at org.apache.hadoop.hbase.quotas.SnapshotQuotaObserverChore._chore(SnapshotQuotaObserverChore.java:118) at org.apache.hadoop.hbase.quotas.SnapshotQuotaObserverChore.chore(SnapshotQuotaObserverChore.java:93) at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:107) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-07-13 06:00:13,011 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-07-13 06:00:13,012 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=testNamespacesInheritSnapshotSize3_snapshot1 table=ns2:testNamespacesInheritSnapshotSize3 type=FLUSH ttl=0 } is done 2023-07-13 06:00:13,013 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotManager(404): Snapshoting '{ ss=testNamespacesInheritSnapshotSize3_snapshot1 table=ns2:testNamespacesInheritSnapshotSize3 type=FLUSH ttl=0 }' is still in progress! 2023-07-13 06:00:13,013 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2428): (#6) Sleeping: 2000ms while waiting for snapshot completion. 2023-07-13 06:00:13,041 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.TakeSnapshotHandler(229): Snapshot testNamespacesInheritSnapshotSize3_snapshot1 of table ns2:testNamespacesInheritSnapshotSize3 completed 2023-07-13 06:00:13,041 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.hbase-snapshot/.tmp/testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:13,041 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.hbase-snapshot/.tmp/testNamespacesInheritSnapshotSize3_snapshot1 2023-07-13 06:00:13,041 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running FLUSH table snapshot testNamespacesInheritSnapshotSize3_snapshot1 C_M_SNAPSHOT_TABLE on table ns2:testNamespacesInheritSnapshotSize3 at 1689228010900Consolidate snapshot: testNamespacesInheritSnapshotSize3_snapshot1 at 1689228011780 (+880 ms)Loading Region manifests for testNamespacesInheritSnapshotSize3_snapshot1 at 1689228011780Writing data manifest for testNamespacesInheritSnapshotSize3_snapshot1 at 1689228011783 (+3 ms)Verifying snapshot: testNamespacesInheritSnapshotSize3_snapshot1 at 1689228012197 (+414 ms)Snapshot testNamespacesInheritSnapshotSize3_snapshot1 of table ns2:testNamespacesInheritSnapshotSize3 completed at 1689228013041 (+844 ms) 2023-07-13 06:00:13,043 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=25, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=ns2:testNamespacesInheritSnapshotSize3, type=SHARED 2023-07-13 06:00:13,044 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=ns2:testNamespacesInheritSnapshotSize3, type=SHARED in 2.1430 sec 2023-07-13 06:00:13,275 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x121da4ef to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:13,280 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2764bdf6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:13,282 DEBUG [hconnection-0x429a88ce-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:13,283 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:37678, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:14,234 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x1624ff10 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:14,245 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@485b8f7e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:14,247 DEBUG [hconnection-0x74cbee75-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:14,250 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:37690, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:15,014 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-07-13 06:00:15,016 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=testNamespacesInheritSnapshotSize3_snapshot1 table=ns2:testNamespacesInheritSnapshotSize3 type=FLUSH ttl=0 } is done 2023-07-13 06:00:15,016 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotManager(401): Snapshot '{ ss=testNamespacesInheritSnapshotSize3_snapshot1 table=ns2:testNamespacesInheritSnapshotSize3 type=FLUSH ttl=0 }' has completed, notifying client. 2023-07-13 06:00:15,016 INFO [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots(238): Write more data 2023-07-13 06:00:15,276 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests(539): Data was written to HBase 2023-07-13 06:00:15,279 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-13 06:00:15,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] procedure.ProcedureCoordinator(143): Procedure ns2:testNamespacesInheritSnapshotSize3 was in running list but was completed. Accepting new attempt. 2023-07-13 06:00:15,282 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] procedure.ProcedureCoordinator(165): Submitting procedure ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,282 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns2:testNamespacesInheritSnapshotSize3' 2023-07-13 06:00:15,282 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-13 06:00:15,282 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns2:testNamespacesInheritSnapshotSize3' starting 'acquire' 2023-07-13 06:00:15,282 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns2:testNamespacesInheritSnapshotSize3', kicking off acquire phase on members. 2023-07-13 06:00:15,283 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,283 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,285 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-13 06:00:15,285 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:15,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-13 06:00:15,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-13 06:00:15,285 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:15,285 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-13 06:00:15,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,286 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-13 06:00:15,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,286 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,287 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,287 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns2:testNamespacesInheritSnapshotSize3' with timeout 60000ms 2023-07-13 06:00:15,287 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-13 06:00:15,287 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns2:testNamespacesInheritSnapshotSize3' starting 'acquire' stage 2023-07-13 06:00:15,288 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-13 06:00:15,288 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-13 06:00:15,288 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f. 2023-07-13 06:00:15,288 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f. started... 2023-07-13 06:00:15,288 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing b6e943f7ce0da690354636beb379d14f 1/1 column families, dataSize=2.22 MB heapSize=2.81 MB 2023-07-13 06:00:15,302 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x105895c2 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:15,310 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@564af44d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:15,313 DEBUG [hconnection-0x2ba5ad66-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:15,314 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:37704, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:15,725 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.22 MB at sequenceid=331 (bloomFilter=true), to=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/ns2/testNamespacesInheritSnapshotSize3/b6e943f7ce0da690354636beb379d14f/.tmp/f1/4034768d0f9844b486ba3031724fed67 2023-07-13 06:00:15,738 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/ns2/testNamespacesInheritSnapshotSize3/b6e943f7ce0da690354636beb379d14f/.tmp/f1/4034768d0f9844b486ba3031724fed67 as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/ns2/testNamespacesInheritSnapshotSize3/b6e943f7ce0da690354636beb379d14f/f1/4034768d0f9844b486ba3031724fed67 2023-07-13 06:00:15,745 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/ns2/testNamespacesInheritSnapshotSize3/b6e943f7ce0da690354636beb379d14f/f1/4034768d0f9844b486ba3031724fed67, entries=8192, sequenceid=331, filesize=2.3 M 2023-07-13 06:00:15,746 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.22 MB/2325418, heapSize ~2.81 MB/2949360, currentSize=0 B/0 for b6e943f7ce0da690354636beb379d14f in 458ms, sequenceid=331, compaction requested=false 2023-07-13 06:00:15,746 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b6e943f7ce0da690354636beb379d14f: 2023-07-13 06:00:15,746 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f. 2023-07-13 06:00:15,747 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-13 06:00:15,747 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-13 06:00:15,747 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:15,747 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns2:testNamespacesInheritSnapshotSize3' locally acquired 2023-07-13 06:00:15,747 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,35187,1689227983701' joining acquired barrier for procedure (ns2:testNamespacesInheritSnapshotSize3) in zk 2023-07-13 06:00:15,749 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,749 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:15,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:15,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:15,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:15,750 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,750 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns2:testNamespacesInheritSnapshotSize3' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-13 06:00:15,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:15,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:15,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:15,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:15,752 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,35187,1689227983701' joining acquired barrier for procedure 'ns2:testNamespacesInheritSnapshotSize3' on coordinator 2023-07-13 06:00:15,752 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1b1775b7[Count = 0] remaining members to acquire global barrier 2023-07-13 06:00:15,752 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns2:testNamespacesInheritSnapshotSize3' starting 'in-barrier' execution. 2023-07-13 06:00:15,752 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,753 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,753 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns2:testNamespacesInheritSnapshotSize3' received 'reached' from coordinator. 2023-07-13 06:00:15,753 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns2:testNamespacesInheritSnapshotSize3' locally completed 2023-07-13 06:00:15,753 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns2:testNamespacesInheritSnapshotSize3' completed for member 'jenkins-hbase20.apache.org,35187,1689227983701' in zk 2023-07-13 06:00:15,753 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:15,753 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-13 06:00:15,755 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns2:testNamespacesInheritSnapshotSize3' has notified controller of completion 2023-07-13 06:00:15,755 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-13 06:00:15,755 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns2:testNamespacesInheritSnapshotSize3' completed. 2023-07-13 06:00:15,755 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:15,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:15,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:15,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:15,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:15,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:15,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:15,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:15,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:15,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns2:testNamespacesInheritSnapshotSize3' member 'jenkins-hbase20.apache.org,35187,1689227983701': 2023-07-13 06:00:15,760 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,35187,1689227983701' released barrier for procedure'ns2:testNamespacesInheritSnapshotSize3', counting down latch. Waiting for 0 more 2023-07-13 06:00:15,760 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns2:testNamespacesInheritSnapshotSize3' execution completed 2023-07-13 06:00:15,760 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-13 06:00:15,760 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-13 06:00:15,760 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,760 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns2:testNamespacesInheritSnapshotSize3including nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-13 06:00:15,762 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,762 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:15,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:15,762 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-13 06:00:15,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:15,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-13 06:00:15,763 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:15,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-13 06:00:15,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:15,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:15,764 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:15,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:15,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:15,765 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:15,765 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,766 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-13 06:00:15,766 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-13 06:00:15,766 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-13 06:00:15,766 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-13 06:00:15,766 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:15,766 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns2:testNamespacesInheritSnapshotSize3' 2023-07-13 06:00:15,766 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-13 06:00:15,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-13 06:00:15,766 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,766 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,766 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,767 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns2:testNamespacesInheritSnapshotSize3'' to complete. (max 20000 ms per retry) 2023-07-13 06:00:15,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-13 06:00:15,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-13 06:00:15,767 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-13 06:00:15,867 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-13 06:00:15,868 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-13 06:00:15,869 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests(542): Data flushed to disk 2023-07-13 06:00:15,869 INFO [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots(240): Flush the table 2023-07-13 06:00:15,873 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-13 06:00:15,875 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] procedure.ProcedureCoordinator(143): Procedure ns2:testNamespacesInheritSnapshotSize3 was in running list but was completed. Accepting new attempt. 2023-07-13 06:00:15,875 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] procedure.ProcedureCoordinator(165): Submitting procedure ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,875 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns2:testNamespacesInheritSnapshotSize3' 2023-07-13 06:00:15,876 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-13 06:00:15,876 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns2:testNamespacesInheritSnapshotSize3' starting 'acquire' 2023-07-13 06:00:15,876 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns2:testNamespacesInheritSnapshotSize3', kicking off acquire phase on members. 2023-07-13 06:00:15,877 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,877 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,878 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-13 06:00:15,879 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:15,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-13 06:00:15,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-13 06:00:15,879 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:15,879 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-13 06:00:15,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,880 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-13 06:00:15,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,880 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,881 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns2:testNamespacesInheritSnapshotSize3 is still present, removing 2023-07-13 06:00:15,881 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,881 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns2:testNamespacesInheritSnapshotSize3' with timeout 60000ms 2023-07-13 06:00:15,881 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-13 06:00:15,881 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns2:testNamespacesInheritSnapshotSize3' starting 'acquire' stage 2023-07-13 06:00:15,881 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-13 06:00:15,881 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-13 06:00:15,881 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f. 2023-07-13 06:00:15,882 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f. started... 2023-07-13 06:00:15,882 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for b6e943f7ce0da690354636beb379d14f: 2023-07-13 06:00:15,882 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f. 2023-07-13 06:00:15,882 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-13 06:00:15,882 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-13 06:00:15,882 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:15,882 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns2:testNamespacesInheritSnapshotSize3' locally acquired 2023-07-13 06:00:15,882 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,35187,1689227983701' joining acquired barrier for procedure (ns2:testNamespacesInheritSnapshotSize3) in zk 2023-07-13 06:00:15,883 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:15,883 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:15,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:15,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:15,883 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,884 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns2:testNamespacesInheritSnapshotSize3' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-13 06:00:15,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:15,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:15,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:15,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:15,885 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,35187,1689227983701' joining acquired barrier for procedure 'ns2:testNamespacesInheritSnapshotSize3' on coordinator 2023-07-13 06:00:15,885 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7d167d18[Count = 0] remaining members to acquire global barrier 2023-07-13 06:00:15,885 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns2:testNamespacesInheritSnapshotSize3' starting 'in-barrier' execution. 2023-07-13 06:00:15,885 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,886 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,886 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns2:testNamespacesInheritSnapshotSize3' received 'reached' from coordinator. 2023-07-13 06:00:15,886 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns2:testNamespacesInheritSnapshotSize3' locally completed 2023-07-13 06:00:15,886 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns2:testNamespacesInheritSnapshotSize3' completed for member 'jenkins-hbase20.apache.org,35187,1689227983701' in zk 2023-07-13 06:00:15,886 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:15,886 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-13 06:00:15,887 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:15,887 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns2:testNamespacesInheritSnapshotSize3' has notified controller of completion 2023-07-13 06:00:15,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:15,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:15,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:15,887 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-13 06:00:15,887 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns2:testNamespacesInheritSnapshotSize3' completed. 2023-07-13 06:00:15,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:15,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:15,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:15,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:15,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:15,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns2:testNamespacesInheritSnapshotSize3' member 'jenkins-hbase20.apache.org,35187,1689227983701': 2023-07-13 06:00:15,891 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,35187,1689227983701' released barrier for procedure'ns2:testNamespacesInheritSnapshotSize3', counting down latch. Waiting for 0 more 2023-07-13 06:00:15,891 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns2:testNamespacesInheritSnapshotSize3' execution completed 2023-07-13 06:00:15,891 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-13 06:00:15,891 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-13 06:00:15,891 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,891 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns2:testNamespacesInheritSnapshotSize3including nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-13 06:00:15,893 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,893 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:15,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:15,893 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-13 06:00:15,893 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:15,894 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-13 06:00:15,894 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:15,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-13 06:00:15,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:15,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:15,895 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:15,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:15,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:15,898 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:15,898 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,898 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-13 06:00:15,898 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,898 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-13 06:00:15,898 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-13 06:00:15,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-13 06:00:15,898 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns2:testNamespacesInheritSnapshotSize3' 2023-07-13 06:00:15,898 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-13 06:00:15,898 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-13 06:00:15,898 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:15,899 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,899 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,899 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-13 06:00:15,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-13 06:00:15,899 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns2:testNamespacesInheritSnapshotSize3 2023-07-13 06:00:15,899 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns2:testNamespacesInheritSnapshotSize3'' to complete. (max 20000 ms per retry) 2023-07-13 06:00:15,899 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-13 06:00:15,999 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-13 06:00:16,002 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-13 06:00:16,003 INFO [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots(242): Synchronously compacting the table 2023-07-13 06:00:16,003 DEBUG [Listener at localhost.localdomain/45717] compactions.SortedCompactionPolicy(75): Selecting compaction from 2 store files, 0 compacting, 2 eligible, 16 blocking 2023-07-13 06:00:16,003 DEBUG [Listener at localhost.localdomain/45717] regionserver.HStore(1912): b6e943f7ce0da690354636beb379d14f/f1 is initiating major compaction (all files) 2023-07-13 06:00:16,004 INFO [Listener at localhost.localdomain/45717] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-13 06:00:16,004 INFO [Listener at localhost.localdomain/45717] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-13 06:00:16,005 INFO [Listener at localhost.localdomain/45717] regionserver.HRegion(2259): Starting compaction of b6e943f7ce0da690354636beb379d14f/f1 in ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f. 2023-07-13 06:00:16,005 INFO [Listener at localhost.localdomain/45717] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/ns2/testNamespacesInheritSnapshotSize3/b6e943f7ce0da690354636beb379d14f/f1/30caf53d6cbf4d35bbd7ff6f186d8472, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/ns2/testNamespacesInheritSnapshotSize3/b6e943f7ce0da690354636beb379d14f/f1/4034768d0f9844b486ba3031724fed67] into tmpdir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/ns2/testNamespacesInheritSnapshotSize3/b6e943f7ce0da690354636beb379d14f/.tmp, totalSize=4.5 M 2023-07-13 06:00:16,007 DEBUG [Listener at localhost.localdomain/45717] compactions.Compactor(207): Compacting 30caf53d6cbf4d35bbd7ff6f186d8472, keycount=8192, bloomtype=ROW, size=2.3 M, encoding=NONE, compression=NONE, seqNum=165, earliestPutTs=1689228005854 2023-07-13 06:00:16,009 DEBUG [Listener at localhost.localdomain/45717] compactions.Compactor(207): Compacting 4034768d0f9844b486ba3031724fed67, keycount=8192, bloomtype=ROW, size=2.3 M, encoding=NONE, compression=NONE, seqNum=331, earliestPutTs=1689228015019 2023-07-13 06:00:16,133 INFO [Listener at localhost.localdomain/45717] throttle.PressureAwareThroughputController(145): b6e943f7ce0da690354636beb379d14f#f1#compaction#5 average throughput is 20.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-13 06:00:16,307 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x6a473778 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:16,312 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@78cd3cae, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:16,314 DEBUG [hconnection-0x3bf70059-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:16,316 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:37706, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:16,373 INFO [master/jenkins-hbase20:0.Chore.1] quotas.QuotaObserverChore(402): ns2:testNamespacesInheritSnapshotSize3 moving into observance of namespace space quota 2023-07-13 06:00:16,552 DEBUG [Listener at localhost.localdomain/45717] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/ns2/testNamespacesInheritSnapshotSize3/b6e943f7ce0da690354636beb379d14f/.tmp/f1/1ff05e64aed043c2bf68fa49655f6c5b as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/ns2/testNamespacesInheritSnapshotSize3/b6e943f7ce0da690354636beb379d14f/f1/1ff05e64aed043c2bf68fa49655f6c5b 2023-07-13 06:00:16,560 INFO [Listener at localhost.localdomain/45717] regionserver.HStore(1652): Completed major compaction of 2 (all) file(s) in b6e943f7ce0da690354636beb379d14f/f1 of b6e943f7ce0da690354636beb379d14f into 1ff05e64aed043c2bf68fa49655f6c5b(size=2.3 M), total size for store is 2.3 M. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-13 06:00:16,560 DEBUG [Listener at localhost.localdomain/45717] regionserver.HRegion(2289): Compaction status journal for b6e943f7ce0da690354636beb379d14f: 2023-07-13 06:00:16,560 INFO [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots(248): Waiting for the region reports to reflect the correct size, between (1585152, 2609152) 2023-07-13 06:00:16,561 INFO [Listener at localhost.localdomain/45717] hbase.Waiter(180): Waiting up to [30,000] milli-secs(wait.for.ratio=[1]) 2023-07-13 06:00:16,568 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$7(254): Master observed table sizes from region size reports: {hbase:meta=0, hbase:namespace=0, testRematerializedTablesDoNoInheritSpace0=4729212, testRematerializedTablesDoNoInheritSpace1=2359672, ns2:testNamespacesInheritSnapshotSize3=4729263, hbase:quota=0} 2023-07-13 06:00:17,069 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$7(254): Master observed table sizes from region size reports: {hbase:meta=0, hbase:namespace=0, testRematerializedTablesDoNoInheritSpace0=4729212, testRematerializedTablesDoNoInheritSpace1=2359672, ns2:testNamespacesInheritSnapshotSize3=4729263, hbase:quota=0} 2023-07-13 06:00:17,280 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x5d5530f7 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:17,285 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@21689e2a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:17,286 DEBUG [hconnection-0x4d1db452-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:17,289 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:37720, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:17,572 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$7(254): Master observed table sizes from region size reports: {hbase:meta=0, hbase:namespace=0, testRematerializedTablesDoNoInheritSpace0=4729212, testRematerializedTablesDoNoInheritSpace1=2359672, ns2:testNamespacesInheritSnapshotSize3=2369769, hbase:quota=0} 2023-07-13 06:00:17,573 INFO [Listener at localhost.localdomain/45717] hbase.Waiter(180): Waiting up to [15,000] milli-secs(wait.for.ratio=[1]) 2023-07-13 06:00:17,573 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$15(466): Last observed size=-1 2023-07-13 06:00:18,075 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$15(466): Last observed size=2369769 2023-07-13 06:00:18,237 INFO [master/jenkins-hbase20:0.Chore.1] quotas.QuotaObserverChore(402): ns2:testNamespacesInheritSnapshotSize3 moving into observance of namespace space quota 2023-07-13 06:00:18,239 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x0e7d55b9 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:18,244 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3071f925, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:18,246 DEBUG [hconnection-0x5bb98c19-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:18,247 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:37722, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:18,578 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$15(466): Last observed size=2369769 2023-07-13 06:00:19,079 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$15(466): Last observed size=2369769 2023-07-13 06:00:19,239 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x5b1b9523 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:19,243 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6a4507a5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:19,247 DEBUG [hconnection-0x887f24f-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:19,249 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:56432, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:19,582 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$15(466): Last observed size=2369769 2023-07-13 06:00:20,084 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$15(466): Last observed size=2369769 2023-07-13 06:00:20,086 INFO [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots(267): Final observed size of table: 2369769 2023-07-13 06:00:20,087 INFO [Listener at localhost.localdomain/45717] hbase.Waiter(180): Waiting up to [20,000] milli-secs(wait.for.ratio=[1]) 2023-07-13 06:00:20,088 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for ns2: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=2.26 MB/1 GB] 2023-07-13 06:00:20,088 INFO [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots(282): Expecting namespace usage to be 2359562 + 2369769 = 4729331 2023-07-13 06:00:20,089 INFO [Listener at localhost.localdomain/45717] hbase.Waiter(180): Waiting up to [30,000] milli-secs(wait.for.ratio=[1]) 2023-07-13 06:00:20,091 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for ns2: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=2.26 MB/1 GB] 2023-07-13 06:00:20,091 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$9(288): Checking for 4729331 == 2369769 2023-07-13 06:00:20,234 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x66be4eee to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:20,238 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@79228cc0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:20,240 DEBUG [hconnection-0x1b7861e2-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:20,242 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:56446, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:20,300 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/ns2/testNamespacesInheritSnapshotSize3/b6e943f7ce0da690354636beb379d14f/f1/30caf53d6cbf4d35bbd7ff6f186d8472, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/ns2/testNamespacesInheritSnapshotSize3/b6e943f7ce0da690354636beb379d14f/f1/4034768d0f9844b486ba3031724fed67] to archive 2023-07-13 06:00:20,301 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-07-13 06:00:20,303 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/ns2/testNamespacesInheritSnapshotSize3/b6e943f7ce0da690354636beb379d14f/f1/30caf53d6cbf4d35bbd7ff6f186d8472 to hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/archive/data/ns2/testNamespacesInheritSnapshotSize3/b6e943f7ce0da690354636beb379d14f/f1/30caf53d6cbf4d35bbd7ff6f186d8472 2023-07-13 06:00:20,304 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/ns2/testNamespacesInheritSnapshotSize3/b6e943f7ce0da690354636beb379d14f/f1/4034768d0f9844b486ba3031724fed67 to hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/archive/data/ns2/testNamespacesInheritSnapshotSize3/b6e943f7ce0da690354636beb379d14f/f1/4034768d0f9844b486ba3031724fed67 2023-07-13 06:00:21,093 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for ns2: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=2.26 MB/1 GB] 2023-07-13 06:00:21,095 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$9(288): Checking for 4729331 == 2369769 2023-07-13 06:00:21,281 INFO [master/jenkins-hbase20:0.Chore.1] quotas.QuotaObserverChore(402): ns2:testNamespacesInheritSnapshotSize3 moving into observance of namespace space quota 2023-07-13 06:00:21,283 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x62bd7799 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:21,288 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4bfd3c93, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:21,290 DEBUG [hconnection-0x2c51a949-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:21,291 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:56454, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:22,097 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for ns2: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=4.51 MB/1 GB] 2023-07-13 06:00:22,098 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$9(288): Checking for 4729331 == 4729331 2023-07-13 06:00:22,122 INFO [Listener at localhost.localdomain/45717] hbase.ResourceChecker(175): after: quotas.TestSpaceQuotasWithSnapshots#testNamespacesInheritSnapshotSize Thread=259 (was 271), OpenFileDescriptor=557 (was 580), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=158 (was 174), ProcessCount=166 (was 168), AvailableMemoryMB=6745 (was 7099) 2023-07-13 06:00:22,135 INFO [Listener at localhost.localdomain/45717] hbase.ResourceChecker(147): before: quotas.TestSpaceQuotasWithSnapshots#testTablesWithSnapshots Thread=259, OpenFileDescriptor=557, MaxFileDescriptor=60000, SystemLoadAverage=158, ProcessCount=166, AvailableMemoryMB=6745 2023-07-13 06:00:22,140 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.HMaster$4(2112): Client=jenkins//148.251.75.209 create 'testTablesWithSnapshots4', {NAME => 'f1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-13 06:00:22,141 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] procedure2.ProcedureExecutor(1029): Stored pid=26, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testTablesWithSnapshots4 2023-07-13 06:00:22,142 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=26, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testTablesWithSnapshots4 execute state=CREATE_TABLE_PRE_OPERATION 2023-07-13 06:00:22,143 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(700): Client=jenkins//148.251.75.209 procedure request for creating table: namespace: "default" qualifier: "testTablesWithSnapshots4" procId is: 26 2023-07-13 06:00:22,143 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=26, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testTablesWithSnapshots4 execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-13 06:00:22,144 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1230): Checking to see if procedure is done pid=26 2023-07-13 06:00:22,149 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testTablesWithSnapshots4/71f62f4fe1b8de992835ee80c072a8c8 2023-07-13 06:00:22,149 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testTablesWithSnapshots4/b06e3484b8c05bbdc046fc6b5386dddc 2023-07-13 06:00:22,150 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testTablesWithSnapshots4/ea36c09b963764399342ace600417093 2023-07-13 06:00:22,150 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testTablesWithSnapshots4/41c84bc63b9629db7bd4a197ebc93b69 2023-07-13 06:00:22,150 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testTablesWithSnapshots4/2994958daca2f80926fd1799582c2b37 2023-07-13 06:00:22,150 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testTablesWithSnapshots4/e16a1a1ea32dc6c137274851c98ece04 2023-07-13 06:00:22,150 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testTablesWithSnapshots4/0d24aa4917843c73c4cba790560f2b8d 2023-07-13 06:00:22,150 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testTablesWithSnapshots4/87846b634b85323bd205e28cff8a7ccc 2023-07-13 06:00:22,150 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testTablesWithSnapshots4/71f62f4fe1b8de992835ee80c072a8c8 empty. 2023-07-13 06:00:22,150 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testTablesWithSnapshots4/ea36c09b963764399342ace600417093 empty. 2023-07-13 06:00:22,150 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testTablesWithSnapshots4/b06e3484b8c05bbdc046fc6b5386dddc empty. 2023-07-13 06:00:22,150 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testTablesWithSnapshots4/87846b634b85323bd205e28cff8a7ccc empty. 2023-07-13 06:00:22,150 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testTablesWithSnapshots4/41c84bc63b9629db7bd4a197ebc93b69 empty. 2023-07-13 06:00:22,151 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testTablesWithSnapshots4/0d24aa4917843c73c4cba790560f2b8d empty. 2023-07-13 06:00:22,151 DEBUG [HFileArchiver-11] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testTablesWithSnapshots4/e16a1a1ea32dc6c137274851c98ece04 empty. 2023-07-13 06:00:22,151 DEBUG [HFileArchiver-9] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testTablesWithSnapshots4/2994958daca2f80926fd1799582c2b37 empty. 2023-07-13 06:00:22,151 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testTablesWithSnapshots4/0d24aa4917843c73c4cba790560f2b8d 2023-07-13 06:00:22,151 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testTablesWithSnapshots4/186edc75693ac04a787ff0d8059708f3 2023-07-13 06:00:22,151 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testTablesWithSnapshots4/41c84bc63b9629db7bd4a197ebc93b69 2023-07-13 06:00:22,151 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testTablesWithSnapshots4/87846b634b85323bd205e28cff8a7ccc 2023-07-13 06:00:22,151 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testTablesWithSnapshots4/6817724a8c0ba54dffa0384c0e1b4834 2023-07-13 06:00:22,151 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testTablesWithSnapshots4/b06e3484b8c05bbdc046fc6b5386dddc 2023-07-13 06:00:22,151 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testTablesWithSnapshots4/ea36c09b963764399342ace600417093 2023-07-13 06:00:22,151 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testTablesWithSnapshots4/71f62f4fe1b8de992835ee80c072a8c8 2023-07-13 06:00:22,151 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testTablesWithSnapshots4/6817724a8c0ba54dffa0384c0e1b4834 empty. 2023-07-13 06:00:22,151 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testTablesWithSnapshots4/186edc75693ac04a787ff0d8059708f3 empty. 2023-07-13 06:00:22,151 DEBUG [HFileArchiver-11] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testTablesWithSnapshots4/e16a1a1ea32dc6c137274851c98ece04 2023-07-13 06:00:22,151 DEBUG [HFileArchiver-9] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testTablesWithSnapshots4/2994958daca2f80926fd1799582c2b37 2023-07-13 06:00:22,152 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testTablesWithSnapshots4/6817724a8c0ba54dffa0384c0e1b4834 2023-07-13 06:00:22,152 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testTablesWithSnapshots4/186edc75693ac04a787ff0d8059708f3 2023-07-13 06:00:22,152 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testTablesWithSnapshots4 regions 2023-07-13 06:00:22,226 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x7a6b06d6 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:22,233 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@79fa4ef2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:22,237 DEBUG [hconnection-0x43c80256-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:22,239 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:56458, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:22,245 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1230): Checking to see if procedure is done pid=26 2023-07-13 06:00:22,446 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1230): Checking to see if procedure is done pid=26 2023-07-13 06:00:22,566 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testTablesWithSnapshots4/.tabledesc/.tableinfo.0000000001 2023-07-13 06:00:22,569 INFO [RegionOpenAndInit-testTablesWithSnapshots4-pool-0] regionserver.HRegion(7675): creating {ENCODED => 71f62f4fe1b8de992835ee80c072a8c8, NAME => 'testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8.', STARTKEY => '', ENDKEY => '0'}, tableDescriptor='testTablesWithSnapshots4', {NAME => 'f1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp 2023-07-13 06:00:22,570 INFO [RegionOpenAndInit-testTablesWithSnapshots4-pool-2] regionserver.HRegion(7675): creating {ENCODED => 41c84bc63b9629db7bd4a197ebc93b69, NAME => 'testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testTablesWithSnapshots4', {NAME => 'f1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp 2023-07-13 06:00:22,570 INFO [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(7675): creating {ENCODED => b06e3484b8c05bbdc046fc6b5386dddc, NAME => 'testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc.', STARTKEY => '0', ENDKEY => '1'}, tableDescriptor='testTablesWithSnapshots4', {NAME => 'f1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp 2023-07-13 06:00:22,748 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1230): Checking to see if procedure is done pid=26 2023-07-13 06:00:22,994 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(866): Instantiated testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 06:00:22,995 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(1604): Closing b06e3484b8c05bbdc046fc6b5386dddc, disabling compactions & flushes 2023-07-13 06:00:22,995 INFO [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(1626): Closing region testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc. 2023-07-13 06:00:22,995 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc. 2023-07-13 06:00:22,995 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(1714): Acquired close lock on testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc. after waiting 0 ms 2023-07-13 06:00:22,995 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(1724): Updates disabled for region testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc. 2023-07-13 06:00:22,996 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-0] regionserver.HRegion(866): Instantiated testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 06:00:22,996 INFO [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(1838): Closed testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc. 2023-07-13 06:00:22,996 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(1558): Region close journal for b06e3484b8c05bbdc046fc6b5386dddc: 2023-07-13 06:00:22,996 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-0] regionserver.HRegion(1604): Closing 71f62f4fe1b8de992835ee80c072a8c8, disabling compactions & flushes 2023-07-13 06:00:22,996 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-2] regionserver.HRegion(866): Instantiated testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 06:00:22,997 INFO [RegionOpenAndInit-testTablesWithSnapshots4-pool-0] regionserver.HRegion(1626): Closing region testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8. 2023-07-13 06:00:22,997 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-2] regionserver.HRegion(1604): Closing 41c84bc63b9629db7bd4a197ebc93b69, disabling compactions & flushes 2023-07-13 06:00:22,997 INFO [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(7675): creating {ENCODED => ea36c09b963764399342ace600417093, NAME => 'testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093.', STARTKEY => '2', ENDKEY => '3'}, tableDescriptor='testTablesWithSnapshots4', {NAME => 'f1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp 2023-07-13 06:00:22,997 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8. 2023-07-13 06:00:22,997 INFO [RegionOpenAndInit-testTablesWithSnapshots4-pool-2] regionserver.HRegion(1626): Closing region testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69. 2023-07-13 06:00:22,998 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69. 2023-07-13 06:00:22,998 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-0] regionserver.HRegion(1714): Acquired close lock on testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8. after waiting 1 ms 2023-07-13 06:00:22,998 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-2] regionserver.HRegion(1714): Acquired close lock on testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69. after waiting 0 ms 2023-07-13 06:00:22,998 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-0] regionserver.HRegion(1724): Updates disabled for region testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8. 2023-07-13 06:00:22,998 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-2] regionserver.HRegion(1724): Updates disabled for region testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69. 2023-07-13 06:00:22,998 INFO [RegionOpenAndInit-testTablesWithSnapshots4-pool-0] regionserver.HRegion(1838): Closed testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8. 2023-07-13 06:00:22,999 INFO [RegionOpenAndInit-testTablesWithSnapshots4-pool-2] regionserver.HRegion(1838): Closed testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69. 2023-07-13 06:00:22,999 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-0] regionserver.HRegion(1558): Region close journal for 71f62f4fe1b8de992835ee80c072a8c8: 2023-07-13 06:00:22,999 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-2] regionserver.HRegion(1558): Region close journal for 41c84bc63b9629db7bd4a197ebc93b69: 2023-07-13 06:00:23,000 INFO [RegionOpenAndInit-testTablesWithSnapshots4-pool-0] regionserver.HRegion(7675): creating {ENCODED => 2994958daca2f80926fd1799582c2b37, NAME => 'testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37.', STARTKEY => '3', ENDKEY => '4'}, tableDescriptor='testTablesWithSnapshots4', {NAME => 'f1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp 2023-07-13 06:00:23,000 INFO [RegionOpenAndInit-testTablesWithSnapshots4-pool-2] regionserver.HRegion(7675): creating {ENCODED => 0d24aa4917843c73c4cba790560f2b8d, NAME => 'testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d.', STARTKEY => '4', ENDKEY => '5'}, tableDescriptor='testTablesWithSnapshots4', {NAME => 'f1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp 2023-07-13 06:00:23,240 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x5d3c72df to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:23,246 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7d0615f3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:23,248 DEBUG [hconnection-0x19458e70-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:23,250 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:56462, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:23,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1230): Checking to see if procedure is done pid=26 2023-07-13 06:00:23,411 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(866): Instantiated testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 06:00:23,412 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(1604): Closing ea36c09b963764399342ace600417093, disabling compactions & flushes 2023-07-13 06:00:23,412 INFO [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(1626): Closing region testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093. 2023-07-13 06:00:23,412 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093. 2023-07-13 06:00:23,412 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(1714): Acquired close lock on testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093. after waiting 0 ms 2023-07-13 06:00:23,412 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(1724): Updates disabled for region testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093. 2023-07-13 06:00:23,412 INFO [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(1838): Closed testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093. 2023-07-13 06:00:23,412 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(1558): Region close journal for ea36c09b963764399342ace600417093: 2023-07-13 06:00:23,412 INFO [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(7675): creating {ENCODED => e16a1a1ea32dc6c137274851c98ece04, NAME => 'testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04.', STARTKEY => '5', ENDKEY => '6'}, tableDescriptor='testTablesWithSnapshots4', {NAME => 'f1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp 2023-07-13 06:00:23,415 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-2] regionserver.HRegion(866): Instantiated testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 06:00:23,415 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-0] regionserver.HRegion(866): Instantiated testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 06:00:23,415 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-2] regionserver.HRegion(1604): Closing 0d24aa4917843c73c4cba790560f2b8d, disabling compactions & flushes 2023-07-13 06:00:23,415 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-0] regionserver.HRegion(1604): Closing 2994958daca2f80926fd1799582c2b37, disabling compactions & flushes 2023-07-13 06:00:23,416 INFO [RegionOpenAndInit-testTablesWithSnapshots4-pool-2] regionserver.HRegion(1626): Closing region testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d. 2023-07-13 06:00:23,418 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d. 2023-07-13 06:00:23,418 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-2] regionserver.HRegion(1714): Acquired close lock on testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d. after waiting 0 ms 2023-07-13 06:00:23,416 INFO [RegionOpenAndInit-testTablesWithSnapshots4-pool-0] regionserver.HRegion(1626): Closing region testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37. 2023-07-13 06:00:23,418 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-2] regionserver.HRegion(1724): Updates disabled for region testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d. 2023-07-13 06:00:23,418 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37. 2023-07-13 06:00:23,418 INFO [RegionOpenAndInit-testTablesWithSnapshots4-pool-2] regionserver.HRegion(1838): Closed testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d. 2023-07-13 06:00:23,418 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-0] regionserver.HRegion(1714): Acquired close lock on testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37. after waiting 0 ms 2023-07-13 06:00:23,418 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-2] regionserver.HRegion(1558): Region close journal for 0d24aa4917843c73c4cba790560f2b8d: 2023-07-13 06:00:23,419 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-0] regionserver.HRegion(1724): Updates disabled for region testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37. 2023-07-13 06:00:23,419 INFO [RegionOpenAndInit-testTablesWithSnapshots4-pool-0] regionserver.HRegion(1838): Closed testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37. 2023-07-13 06:00:23,419 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-0] regionserver.HRegion(1558): Region close journal for 2994958daca2f80926fd1799582c2b37: 2023-07-13 06:00:23,420 INFO [RegionOpenAndInit-testTablesWithSnapshots4-pool-0] regionserver.HRegion(7675): creating {ENCODED => 186edc75693ac04a787ff0d8059708f3, NAME => 'testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3.', STARTKEY => '7', ENDKEY => '9'}, tableDescriptor='testTablesWithSnapshots4', {NAME => 'f1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp 2023-07-13 06:00:23,420 INFO [RegionOpenAndInit-testTablesWithSnapshots4-pool-2] regionserver.HRegion(7675): creating {ENCODED => 87846b634b85323bd205e28cff8a7ccc, NAME => 'testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc.', STARTKEY => '6', ENDKEY => '7'}, tableDescriptor='testTablesWithSnapshots4', {NAME => 'f1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp 2023-07-13 06:00:23,827 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(866): Instantiated testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 06:00:23,828 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(1604): Closing e16a1a1ea32dc6c137274851c98ece04, disabling compactions & flushes 2023-07-13 06:00:23,828 INFO [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(1626): Closing region testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04. 2023-07-13 06:00:23,828 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04. 2023-07-13 06:00:23,828 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(1714): Acquired close lock on testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04. after waiting 0 ms 2023-07-13 06:00:23,828 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(1724): Updates disabled for region testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04. 2023-07-13 06:00:23,828 INFO [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(1838): Closed testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04. 2023-07-13 06:00:23,828 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(1558): Region close journal for e16a1a1ea32dc6c137274851c98ece04: 2023-07-13 06:00:23,829 INFO [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(7675): creating {ENCODED => 6817724a8c0ba54dffa0384c0e1b4834, NAME => 'testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834.', STARTKEY => '9', ENDKEY => ''}, tableDescriptor='testTablesWithSnapshots4', {NAME => 'f1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp 2023-07-13 06:00:23,833 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-0] regionserver.HRegion(866): Instantiated testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 06:00:23,834 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-0] regionserver.HRegion(1604): Closing 186edc75693ac04a787ff0d8059708f3, disabling compactions & flushes 2023-07-13 06:00:23,834 INFO [RegionOpenAndInit-testTablesWithSnapshots4-pool-0] regionserver.HRegion(1626): Closing region testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3. 2023-07-13 06:00:23,834 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3. 2023-07-13 06:00:23,834 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-0] regionserver.HRegion(1714): Acquired close lock on testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3. after waiting 0 ms 2023-07-13 06:00:23,834 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-2] regionserver.HRegion(866): Instantiated testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 06:00:23,834 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-0] regionserver.HRegion(1724): Updates disabled for region testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3. 2023-07-13 06:00:23,834 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-2] regionserver.HRegion(1604): Closing 87846b634b85323bd205e28cff8a7ccc, disabling compactions & flushes 2023-07-13 06:00:23,834 INFO [RegionOpenAndInit-testTablesWithSnapshots4-pool-0] regionserver.HRegion(1838): Closed testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3. 2023-07-13 06:00:23,835 INFO [RegionOpenAndInit-testTablesWithSnapshots4-pool-2] regionserver.HRegion(1626): Closing region testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc. 2023-07-13 06:00:23,835 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-0] regionserver.HRegion(1558): Region close journal for 186edc75693ac04a787ff0d8059708f3: 2023-07-13 06:00:23,835 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc. 2023-07-13 06:00:23,835 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-2] regionserver.HRegion(1714): Acquired close lock on testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc. after waiting 0 ms 2023-07-13 06:00:23,835 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-2] regionserver.HRegion(1724): Updates disabled for region testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc. 2023-07-13 06:00:23,835 INFO [RegionOpenAndInit-testTablesWithSnapshots4-pool-2] regionserver.HRegion(1838): Closed testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc. 2023-07-13 06:00:23,835 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-2] regionserver.HRegion(1558): Region close journal for 87846b634b85323bd205e28cff8a7ccc: 2023-07-13 06:00:24,225 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x49335a62 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:24,230 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@73bc1ce0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:24,231 DEBUG [hconnection-0x5a40baa4-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:24,232 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:56472, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:24,240 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(866): Instantiated testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 06:00:24,240 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(1604): Closing 6817724a8c0ba54dffa0384c0e1b4834, disabling compactions & flushes 2023-07-13 06:00:24,240 INFO [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(1626): Closing region testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834. 2023-07-13 06:00:24,240 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834. 2023-07-13 06:00:24,240 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(1714): Acquired close lock on testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834. after waiting 0 ms 2023-07-13 06:00:24,240 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(1724): Updates disabled for region testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834. 2023-07-13 06:00:24,240 INFO [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(1838): Closed testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834. 2023-07-13 06:00:24,240 DEBUG [RegionOpenAndInit-testTablesWithSnapshots4-pool-1] regionserver.HRegion(1558): Region close journal for 6817724a8c0ba54dffa0384c0e1b4834: 2023-07-13 06:00:24,242 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=26, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testTablesWithSnapshots4 execute state=CREATE_TABLE_ADD_TO_META 2023-07-13 06:00:24,243 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc.","families":{"info":[{"qualifier":"regioninfo","vlen":60,"tag":[],"timestamp":"1689228024243"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689228024243"}]},"ts":"1689228024243"} 2023-07-13 06:00:24,243 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8.","families":{"info":[{"qualifier":"regioninfo","vlen":59,"tag":[],"timestamp":"1689228024243"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689228024243"}]},"ts":"1689228024243"} 2023-07-13 06:00:24,243 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69.","families":{"info":[{"qualifier":"regioninfo","vlen":60,"tag":[],"timestamp":"1689228024243"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689228024243"}]},"ts":"1689228024243"} 2023-07-13 06:00:24,244 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093.","families":{"info":[{"qualifier":"regioninfo","vlen":60,"tag":[],"timestamp":"1689228024243"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689228024243"}]},"ts":"1689228024243"} 2023-07-13 06:00:24,244 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d.","families":{"info":[{"qualifier":"regioninfo","vlen":60,"tag":[],"timestamp":"1689228024243"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689228024243"}]},"ts":"1689228024243"} 2023-07-13 06:00:24,244 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37.","families":{"info":[{"qualifier":"regioninfo","vlen":60,"tag":[],"timestamp":"1689228024243"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689228024243"}]},"ts":"1689228024243"} 2023-07-13 06:00:24,244 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04.","families":{"info":[{"qualifier":"regioninfo","vlen":60,"tag":[],"timestamp":"1689228024243"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689228024243"}]},"ts":"1689228024243"} 2023-07-13 06:00:24,244 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3.","families":{"info":[{"qualifier":"regioninfo","vlen":60,"tag":[],"timestamp":"1689228024243"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689228024243"}]},"ts":"1689228024243"} 2023-07-13 06:00:24,244 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc.","families":{"info":[{"qualifier":"regioninfo","vlen":60,"tag":[],"timestamp":"1689228024243"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689228024243"}]},"ts":"1689228024243"} 2023-07-13 06:00:24,244 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834.","families":{"info":[{"qualifier":"regioninfo","vlen":59,"tag":[],"timestamp":"1689228024243"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689228024243"}]},"ts":"1689228024243"} 2023-07-13 06:00:24,246 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 10 regions to meta. 2023-07-13 06:00:24,247 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=26, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testTablesWithSnapshots4 execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-13 06:00:24,247 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testTablesWithSnapshots4","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689228024247"}]},"ts":"1689228024247"} 2023-07-13 06:00:24,248 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testTablesWithSnapshots4, state=ENABLING in hbase:meta 2023-07-13 06:00:24,250 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=71f62f4fe1b8de992835ee80c072a8c8, ASSIGN}, {pid=28, ppid=26, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=b06e3484b8c05bbdc046fc6b5386dddc, ASSIGN}, {pid=29, ppid=26, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=41c84bc63b9629db7bd4a197ebc93b69, ASSIGN}, {pid=30, ppid=26, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=ea36c09b963764399342ace600417093, ASSIGN}, {pid=31, ppid=26, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=2994958daca2f80926fd1799582c2b37, ASSIGN}, {pid=32, ppid=26, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=0d24aa4917843c73c4cba790560f2b8d, ASSIGN}, {pid=33, ppid=26, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=e16a1a1ea32dc6c137274851c98ece04, ASSIGN}, {pid=34, ppid=26, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=87846b634b85323bd205e28cff8a7ccc, ASSIGN}, {pid=35, ppid=26, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=186edc75693ac04a787ff0d8059708f3, ASSIGN}, {pid=36, ppid=26, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=6817724a8c0ba54dffa0384c0e1b4834, ASSIGN}] 2023-07-13 06:00:24,252 WARN [master/jenkins-hbase20:0.Chore.1] util.FSUtils$2(1259): Skipping region because it has no family dirs: hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/0d24aa4917843c73c4cba790560f2b8d 2023-07-13 06:00:24,253 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=26, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=ea36c09b963764399342ace600417093, ASSIGN 2023-07-13 06:00:24,253 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=26, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=41c84bc63b9629db7bd4a197ebc93b69, ASSIGN 2023-07-13 06:00:24,253 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=26, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=b06e3484b8c05bbdc046fc6b5386dddc, ASSIGN 2023-07-13 06:00:24,253 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=27, ppid=26, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=71f62f4fe1b8de992835ee80c072a8c8, ASSIGN 2023-07-13 06:00:24,254 WARN [master/jenkins-hbase20:0.Chore.1] util.FSUtils$2(1259): Skipping region because it has no family dirs: hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/186edc75693ac04a787ff0d8059708f3 2023-07-13 06:00:24,254 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=26, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=b06e3484b8c05bbdc046fc6b5386dddc, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,35187,1689227983701; forceNewPlan=false, retain=false 2023-07-13 06:00:24,254 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=29, ppid=26, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=41c84bc63b9629db7bd4a197ebc93b69, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,35187,1689227983701; forceNewPlan=false, retain=false 2023-07-13 06:00:24,254 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=27, ppid=26, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=71f62f4fe1b8de992835ee80c072a8c8, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,35187,1689227983701; forceNewPlan=false, retain=false 2023-07-13 06:00:24,254 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=26, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=ea36c09b963764399342ace600417093, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,35187,1689227983701; forceNewPlan=false, retain=false 2023-07-13 06:00:24,255 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1230): Checking to see if procedure is done pid=26 2023-07-13 06:00:24,255 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=26, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=6817724a8c0ba54dffa0384c0e1b4834, ASSIGN 2023-07-13 06:00:24,255 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=35, ppid=26, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=186edc75693ac04a787ff0d8059708f3, ASSIGN 2023-07-13 06:00:24,256 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=33, ppid=26, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=e16a1a1ea32dc6c137274851c98ece04, ASSIGN 2023-07-13 06:00:24,256 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=34, ppid=26, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=87846b634b85323bd205e28cff8a7ccc, ASSIGN 2023-07-13 06:00:24,256 WARN [master/jenkins-hbase20:0.Chore.1] util.FSUtils$2(1259): Skipping region because it has no family dirs: hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/2994958daca2f80926fd1799582c2b37 2023-07-13 06:00:24,256 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=26, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=0d24aa4917843c73c4cba790560f2b8d, ASSIGN 2023-07-13 06:00:24,256 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=35, ppid=26, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=186edc75693ac04a787ff0d8059708f3, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,35187,1689227983701; forceNewPlan=false, retain=false 2023-07-13 06:00:24,256 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=36, ppid=26, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=6817724a8c0ba54dffa0384c0e1b4834, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,35187,1689227983701; forceNewPlan=false, retain=false 2023-07-13 06:00:24,256 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=34, ppid=26, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=87846b634b85323bd205e28cff8a7ccc, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,35187,1689227983701; forceNewPlan=false, retain=false 2023-07-13 06:00:24,257 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=26, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=0d24aa4917843c73c4cba790560f2b8d, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,35187,1689227983701; forceNewPlan=false, retain=false 2023-07-13 06:00:24,256 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=33, ppid=26, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=e16a1a1ea32dc6c137274851c98ece04, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,35187,1689227983701; forceNewPlan=false, retain=false 2023-07-13 06:00:24,257 WARN [master/jenkins-hbase20:0.Chore.1] util.FSUtils$2(1259): Skipping region because it has no family dirs: hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/41c84bc63b9629db7bd4a197ebc93b69 2023-07-13 06:00:24,257 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=26, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=2994958daca2f80926fd1799582c2b37, ASSIGN 2023-07-13 06:00:24,258 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=26, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=2994958daca2f80926fd1799582c2b37, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,35187,1689227983701; forceNewPlan=false, retain=false 2023-07-13 06:00:24,259 WARN [master/jenkins-hbase20:0.Chore.1] util.FSUtils$2(1259): Skipping region because it has no family dirs: hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/6817724a8c0ba54dffa0384c0e1b4834 2023-07-13 06:00:24,260 WARN [master/jenkins-hbase20:0.Chore.1] util.FSUtils$2(1259): Skipping region because it has no family dirs: hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/71f62f4fe1b8de992835ee80c072a8c8 2023-07-13 06:00:24,260 WARN [master/jenkins-hbase20:0.Chore.1] util.FSUtils$2(1259): Skipping region because it has no family dirs: hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/87846b634b85323bd205e28cff8a7ccc 2023-07-13 06:00:24,261 WARN [master/jenkins-hbase20:0.Chore.1] util.FSUtils$2(1259): Skipping region because it has no family dirs: hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/b06e3484b8c05bbdc046fc6b5386dddc 2023-07-13 06:00:24,262 WARN [master/jenkins-hbase20:0.Chore.1] util.FSUtils$2(1259): Skipping region because it has no family dirs: hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/e16a1a1ea32dc6c137274851c98ece04 2023-07-13 06:00:24,262 WARN [master/jenkins-hbase20:0.Chore.1] util.FSUtils$2(1259): Skipping region because it has no family dirs: hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/ea36c09b963764399342ace600417093 2023-07-13 06:00:24,281 WARN [master/jenkins-hbase20:0.Chore.1] util.FSUtils$2(1259): Skipping region because it has no family dirs: hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/0d24aa4917843c73c4cba790560f2b8d 2023-07-13 06:00:24,282 WARN [master/jenkins-hbase20:0.Chore.1] util.FSUtils$2(1259): Skipping region because it has no family dirs: hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/186edc75693ac04a787ff0d8059708f3 2023-07-13 06:00:24,283 WARN [master/jenkins-hbase20:0.Chore.1] util.FSUtils$2(1259): Skipping region because it has no family dirs: hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/2994958daca2f80926fd1799582c2b37 2023-07-13 06:00:24,283 WARN [master/jenkins-hbase20:0.Chore.1] util.FSUtils$2(1259): Skipping region because it has no family dirs: hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/41c84bc63b9629db7bd4a197ebc93b69 2023-07-13 06:00:24,284 WARN [master/jenkins-hbase20:0.Chore.1] util.FSUtils$2(1259): Skipping region because it has no family dirs: hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/6817724a8c0ba54dffa0384c0e1b4834 2023-07-13 06:00:24,285 WARN [master/jenkins-hbase20:0.Chore.1] util.FSUtils$2(1259): Skipping region because it has no family dirs: hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/71f62f4fe1b8de992835ee80c072a8c8 2023-07-13 06:00:24,285 WARN [master/jenkins-hbase20:0.Chore.1] util.FSUtils$2(1259): Skipping region because it has no family dirs: hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/87846b634b85323bd205e28cff8a7ccc 2023-07-13 06:00:24,286 WARN [master/jenkins-hbase20:0.Chore.1] util.FSUtils$2(1259): Skipping region because it has no family dirs: hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/b06e3484b8c05bbdc046fc6b5386dddc 2023-07-13 06:00:24,286 WARN [master/jenkins-hbase20:0.Chore.1] util.FSUtils$2(1259): Skipping region because it has no family dirs: hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/e16a1a1ea32dc6c137274851c98ece04 2023-07-13 06:00:24,287 WARN [master/jenkins-hbase20:0.Chore.1] util.FSUtils$2(1259): Skipping region because it has no family dirs: hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/ea36c09b963764399342ace600417093 2023-07-13 06:00:24,410 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=b06e3484b8c05bbdc046fc6b5386dddc, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:24,410 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc.","families":{"info":[{"qualifier":"regioninfo","vlen":60,"tag":[],"timestamp":"1689228024410"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689228024410"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689228024410"}]},"ts":"1689228024410"} 2023-07-13 06:00:24,410 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=35 updating hbase:meta row=186edc75693ac04a787ff0d8059708f3, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:24,410 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=0d24aa4917843c73c4cba790560f2b8d, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:24,411 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3.","families":{"info":[{"qualifier":"regioninfo","vlen":60,"tag":[],"timestamp":"1689228024410"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689228024410"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689228024410"}]},"ts":"1689228024410"} 2023-07-13 06:00:24,411 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d.","families":{"info":[{"qualifier":"regioninfo","vlen":60,"tag":[],"timestamp":"1689228024410"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689228024410"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689228024410"}]},"ts":"1689228024410"} 2023-07-13 06:00:24,412 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=ea36c09b963764399342ace600417093, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:24,412 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093.","families":{"info":[{"qualifier":"regioninfo","vlen":60,"tag":[],"timestamp":"1689228024412"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689228024412"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689228024412"}]},"ts":"1689228024412"} 2023-07-13 06:00:24,415 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=28, state=RUNNABLE; OpenRegionProcedure b06e3484b8c05bbdc046fc6b5386dddc, server=jenkins-hbase20.apache.org,35187,1689227983701}] 2023-07-13 06:00:24,415 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=32, state=RUNNABLE; OpenRegionProcedure 0d24aa4917843c73c4cba790560f2b8d, server=jenkins-hbase20.apache.org,35187,1689227983701}] 2023-07-13 06:00:24,415 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=34 updating hbase:meta row=87846b634b85323bd205e28cff8a7ccc, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:24,415 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc.","families":{"info":[{"qualifier":"regioninfo","vlen":60,"tag":[],"timestamp":"1689228024412"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689228024412"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689228024412"}]},"ts":"1689228024412"} 2023-07-13 06:00:24,415 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=37, ppid=35, state=RUNNABLE; OpenRegionProcedure 186edc75693ac04a787ff0d8059708f3, server=jenkins-hbase20.apache.org,35187,1689227983701}] 2023-07-13 06:00:24,418 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=27 updating hbase:meta row=71f62f4fe1b8de992835ee80c072a8c8, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:24,419 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=40, ppid=30, state=RUNNABLE; OpenRegionProcedure ea36c09b963764399342ace600417093, server=jenkins-hbase20.apache.org,35187,1689227983701}] 2023-07-13 06:00:24,419 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=41, ppid=34, state=RUNNABLE; OpenRegionProcedure 87846b634b85323bd205e28cff8a7ccc, server=jenkins-hbase20.apache.org,35187,1689227983701}] 2023-07-13 06:00:24,419 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=41c84bc63b9629db7bd4a197ebc93b69, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:24,419 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8.","families":{"info":[{"qualifier":"regioninfo","vlen":59,"tag":[],"timestamp":"1689228024418"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689228024418"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689228024418"}]},"ts":"1689228024418"} 2023-07-13 06:00:24,419 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69.","families":{"info":[{"qualifier":"regioninfo","vlen":60,"tag":[],"timestamp":"1689228024419"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689228024419"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689228024419"}]},"ts":"1689228024419"} 2023-07-13 06:00:24,419 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=2994958daca2f80926fd1799582c2b37, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:24,420 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37.","families":{"info":[{"qualifier":"regioninfo","vlen":60,"tag":[],"timestamp":"1689228024419"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689228024419"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689228024419"}]},"ts":"1689228024419"} 2023-07-13 06:00:24,420 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=29, state=RUNNABLE; OpenRegionProcedure 41c84bc63b9629db7bd4a197ebc93b69, server=jenkins-hbase20.apache.org,35187,1689227983701}] 2023-07-13 06:00:24,421 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=27, state=RUNNABLE; OpenRegionProcedure 71f62f4fe1b8de992835ee80c072a8c8, server=jenkins-hbase20.apache.org,35187,1689227983701}] 2023-07-13 06:00:24,422 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=44, ppid=31, state=RUNNABLE; OpenRegionProcedure 2994958daca2f80926fd1799582c2b37, server=jenkins-hbase20.apache.org,35187,1689227983701}] 2023-07-13 06:00:24,423 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=6817724a8c0ba54dffa0384c0e1b4834, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:24,423 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834.","families":{"info":[{"qualifier":"regioninfo","vlen":59,"tag":[],"timestamp":"1689228024422"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689228024422"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689228024422"}]},"ts":"1689228024422"} 2023-07-13 06:00:24,423 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=33 updating hbase:meta row=e16a1a1ea32dc6c137274851c98ece04, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:24,423 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04.","families":{"info":[{"qualifier":"regioninfo","vlen":60,"tag":[],"timestamp":"1689228024423"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689228024423"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689228024423"}]},"ts":"1689228024423"} 2023-07-13 06:00:24,424 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=36, state=RUNNABLE; OpenRegionProcedure 6817724a8c0ba54dffa0384c0e1b4834, server=jenkins-hbase20.apache.org,35187,1689227983701}] 2023-07-13 06:00:24,425 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=33, state=RUNNABLE; OpenRegionProcedure e16a1a1ea32dc6c137274851c98ece04, server=jenkins-hbase20.apache.org,35187,1689227983701}] 2023-07-13 06:00:24,572 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc. 2023-07-13 06:00:24,572 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 87846b634b85323bd205e28cff8a7ccc, NAME => 'testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc.', STARTKEY => '6', ENDKEY => '7'} 2023-07-13 06:00:24,572 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testTablesWithSnapshots4 87846b634b85323bd205e28cff8a7ccc 2023-07-13 06:00:24,573 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 06:00:24,573 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 87846b634b85323bd205e28cff8a7ccc 2023-07-13 06:00:24,573 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 87846b634b85323bd205e28cff8a7ccc 2023-07-13 06:00:24,575 INFO [StoreOpener-87846b634b85323bd205e28cff8a7ccc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region 87846b634b85323bd205e28cff8a7ccc 2023-07-13 06:00:24,578 DEBUG [StoreOpener-87846b634b85323bd205e28cff8a7ccc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/87846b634b85323bd205e28cff8a7ccc/f1 2023-07-13 06:00:24,578 DEBUG [StoreOpener-87846b634b85323bd205e28cff8a7ccc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/87846b634b85323bd205e28cff8a7ccc/f1 2023-07-13 06:00:24,578 INFO [StoreOpener-87846b634b85323bd205e28cff8a7ccc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 87846b634b85323bd205e28cff8a7ccc columnFamilyName f1 2023-07-13 06:00:24,579 INFO [StoreOpener-87846b634b85323bd205e28cff8a7ccc-1] regionserver.HStore(310): Store=87846b634b85323bd205e28cff8a7ccc/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-13 06:00:24,580 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/87846b634b85323bd205e28cff8a7ccc 2023-07-13 06:00:24,580 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/87846b634b85323bd205e28cff8a7ccc 2023-07-13 06:00:24,583 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 87846b634b85323bd205e28cff8a7ccc 2023-07-13 06:00:24,585 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/87846b634b85323bd205e28cff8a7ccc/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-13 06:00:24,585 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 87846b634b85323bd205e28cff8a7ccc; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68209113, jitterRate=0.01639498770236969}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-13 06:00:24,585 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 87846b634b85323bd205e28cff8a7ccc: 2023-07-13 06:00:24,586 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc., pid=41, masterSystemTime=1689228024567 2023-07-13 06:00:24,587 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc. 2023-07-13 06:00:24,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc. 2023-07-13 06:00:24,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69. 2023-07-13 06:00:24,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 41c84bc63b9629db7bd4a197ebc93b69, NAME => 'testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69.', STARTKEY => '1', ENDKEY => '2'} 2023-07-13 06:00:24,588 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=34 updating hbase:meta row=87846b634b85323bd205e28cff8a7ccc, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:24,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testTablesWithSnapshots4 41c84bc63b9629db7bd4a197ebc93b69 2023-07-13 06:00:24,588 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc.","families":{"info":[{"qualifier":"regioninfo","vlen":60,"tag":[],"timestamp":"1689228024588"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689228024588"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689228024588"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689228024588"}]},"ts":"1689228024588"} 2023-07-13 06:00:24,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 06:00:24,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 41c84bc63b9629db7bd4a197ebc93b69 2023-07-13 06:00:24,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 41c84bc63b9629db7bd4a197ebc93b69 2023-07-13 06:00:24,589 INFO [StoreOpener-41c84bc63b9629db7bd4a197ebc93b69-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region 41c84bc63b9629db7bd4a197ebc93b69 2023-07-13 06:00:24,591 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=41, resume processing ppid=34 2023-07-13 06:00:24,591 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=41, ppid=34, state=SUCCESS; OpenRegionProcedure 87846b634b85323bd205e28cff8a7ccc, server=jenkins-hbase20.apache.org,35187,1689227983701 in 170 msec 2023-07-13 06:00:24,591 DEBUG [StoreOpener-41c84bc63b9629db7bd4a197ebc93b69-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/41c84bc63b9629db7bd4a197ebc93b69/f1 2023-07-13 06:00:24,591 DEBUG [StoreOpener-41c84bc63b9629db7bd4a197ebc93b69-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/41c84bc63b9629db7bd4a197ebc93b69/f1 2023-07-13 06:00:24,592 INFO [StoreOpener-41c84bc63b9629db7bd4a197ebc93b69-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 41c84bc63b9629db7bd4a197ebc93b69 columnFamilyName f1 2023-07-13 06:00:24,592 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=26, state=SUCCESS; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=87846b634b85323bd205e28cff8a7ccc, ASSIGN in 341 msec 2023-07-13 06:00:24,592 INFO [StoreOpener-41c84bc63b9629db7bd4a197ebc93b69-1] regionserver.HStore(310): Store=41c84bc63b9629db7bd4a197ebc93b69/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-13 06:00:24,593 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/41c84bc63b9629db7bd4a197ebc93b69 2023-07-13 06:00:24,593 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/41c84bc63b9629db7bd4a197ebc93b69 2023-07-13 06:00:24,596 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 41c84bc63b9629db7bd4a197ebc93b69 2023-07-13 06:00:24,598 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/41c84bc63b9629db7bd4a197ebc93b69/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-13 06:00:24,598 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 41c84bc63b9629db7bd4a197ebc93b69; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74425039, jitterRate=0.10901950299739838}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-13 06:00:24,598 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 41c84bc63b9629db7bd4a197ebc93b69: 2023-07-13 06:00:24,599 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69., pid=42, masterSystemTime=1689228024567 2023-07-13 06:00:24,600 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69. 2023-07-13 06:00:24,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69. 2023-07-13 06:00:24,601 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37. 2023-07-13 06:00:24,601 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2994958daca2f80926fd1799582c2b37, NAME => 'testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37.', STARTKEY => '3', ENDKEY => '4'} 2023-07-13 06:00:24,601 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=41c84bc63b9629db7bd4a197ebc93b69, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:24,601 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testTablesWithSnapshots4 2994958daca2f80926fd1799582c2b37 2023-07-13 06:00:24,601 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 06:00:24,601 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69.","families":{"info":[{"qualifier":"regioninfo","vlen":60,"tag":[],"timestamp":"1689228024601"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689228024601"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689228024601"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689228024601"}]},"ts":"1689228024601"} 2023-07-13 06:00:24,601 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 2994958daca2f80926fd1799582c2b37 2023-07-13 06:00:24,601 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 2994958daca2f80926fd1799582c2b37 2023-07-13 06:00:24,603 INFO [StoreOpener-2994958daca2f80926fd1799582c2b37-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region 2994958daca2f80926fd1799582c2b37 2023-07-13 06:00:24,604 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=29 2023-07-13 06:00:24,604 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=29, state=SUCCESS; OpenRegionProcedure 41c84bc63b9629db7bd4a197ebc93b69, server=jenkins-hbase20.apache.org,35187,1689227983701 in 183 msec 2023-07-13 06:00:24,605 DEBUG [StoreOpener-2994958daca2f80926fd1799582c2b37-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/2994958daca2f80926fd1799582c2b37/f1 2023-07-13 06:00:24,605 DEBUG [StoreOpener-2994958daca2f80926fd1799582c2b37-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/2994958daca2f80926fd1799582c2b37/f1 2023-07-13 06:00:24,605 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=26, state=SUCCESS; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=41c84bc63b9629db7bd4a197ebc93b69, ASSIGN in 354 msec 2023-07-13 06:00:24,606 INFO [StoreOpener-2994958daca2f80926fd1799582c2b37-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2994958daca2f80926fd1799582c2b37 columnFamilyName f1 2023-07-13 06:00:24,607 INFO [StoreOpener-2994958daca2f80926fd1799582c2b37-1] regionserver.HStore(310): Store=2994958daca2f80926fd1799582c2b37/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-13 06:00:24,608 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/2994958daca2f80926fd1799582c2b37 2023-07-13 06:00:24,609 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/2994958daca2f80926fd1799582c2b37 2023-07-13 06:00:24,612 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 2994958daca2f80926fd1799582c2b37 2023-07-13 06:00:24,614 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/2994958daca2f80926fd1799582c2b37/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-13 06:00:24,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 2994958daca2f80926fd1799582c2b37; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62021177, jitterRate=-0.0758124440908432}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-13 06:00:24,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 2994958daca2f80926fd1799582c2b37: 2023-07-13 06:00:24,616 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37., pid=44, masterSystemTime=1689228024567 2023-07-13 06:00:24,618 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37. 2023-07-13 06:00:24,618 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37. 2023-07-13 06:00:24,618 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8. 2023-07-13 06:00:24,618 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 71f62f4fe1b8de992835ee80c072a8c8, NAME => 'testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8.', STARTKEY => '', ENDKEY => '0'} 2023-07-13 06:00:24,618 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=2994958daca2f80926fd1799582c2b37, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:24,618 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testTablesWithSnapshots4 71f62f4fe1b8de992835ee80c072a8c8 2023-07-13 06:00:24,618 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 06:00:24,618 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37.","families":{"info":[{"qualifier":"regioninfo","vlen":60,"tag":[],"timestamp":"1689228024618"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689228024618"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689228024618"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689228024618"}]},"ts":"1689228024618"} 2023-07-13 06:00:24,619 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 71f62f4fe1b8de992835ee80c072a8c8 2023-07-13 06:00:24,619 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 71f62f4fe1b8de992835ee80c072a8c8 2023-07-13 06:00:24,621 INFO [StoreOpener-71f62f4fe1b8de992835ee80c072a8c8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region 71f62f4fe1b8de992835ee80c072a8c8 2023-07-13 06:00:24,622 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=44, resume processing ppid=31 2023-07-13 06:00:24,623 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=44, ppid=31, state=SUCCESS; OpenRegionProcedure 2994958daca2f80926fd1799582c2b37, server=jenkins-hbase20.apache.org,35187,1689227983701 in 198 msec 2023-07-13 06:00:24,623 DEBUG [StoreOpener-71f62f4fe1b8de992835ee80c072a8c8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/71f62f4fe1b8de992835ee80c072a8c8/f1 2023-07-13 06:00:24,623 DEBUG [StoreOpener-71f62f4fe1b8de992835ee80c072a8c8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/71f62f4fe1b8de992835ee80c072a8c8/f1 2023-07-13 06:00:24,624 INFO [StoreOpener-71f62f4fe1b8de992835ee80c072a8c8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 71f62f4fe1b8de992835ee80c072a8c8 columnFamilyName f1 2023-07-13 06:00:24,624 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=26, state=SUCCESS; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=2994958daca2f80926fd1799582c2b37, ASSIGN in 372 msec 2023-07-13 06:00:24,624 INFO [StoreOpener-71f62f4fe1b8de992835ee80c072a8c8-1] regionserver.HStore(310): Store=71f62f4fe1b8de992835ee80c072a8c8/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-13 06:00:24,625 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/71f62f4fe1b8de992835ee80c072a8c8 2023-07-13 06:00:24,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/71f62f4fe1b8de992835ee80c072a8c8 2023-07-13 06:00:24,630 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 71f62f4fe1b8de992835ee80c072a8c8 2023-07-13 06:00:24,633 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/71f62f4fe1b8de992835ee80c072a8c8/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-13 06:00:24,634 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 71f62f4fe1b8de992835ee80c072a8c8; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59112545, jitterRate=-0.1191544383764267}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-13 06:00:24,634 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 71f62f4fe1b8de992835ee80c072a8c8: 2023-07-13 06:00:24,635 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8., pid=43, masterSystemTime=1689228024567 2023-07-13 06:00:24,636 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8. 2023-07-13 06:00:24,636 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8. 2023-07-13 06:00:24,636 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc. 2023-07-13 06:00:24,637 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b06e3484b8c05bbdc046fc6b5386dddc, NAME => 'testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc.', STARTKEY => '0', ENDKEY => '1'} 2023-07-13 06:00:24,637 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=27 updating hbase:meta row=71f62f4fe1b8de992835ee80c072a8c8, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:24,637 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testTablesWithSnapshots4 b06e3484b8c05bbdc046fc6b5386dddc 2023-07-13 06:00:24,637 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8.","families":{"info":[{"qualifier":"regioninfo","vlen":59,"tag":[],"timestamp":"1689228024637"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689228024637"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689228024637"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689228024637"}]},"ts":"1689228024637"} 2023-07-13 06:00:24,637 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 06:00:24,637 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for b06e3484b8c05bbdc046fc6b5386dddc 2023-07-13 06:00:24,637 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for b06e3484b8c05bbdc046fc6b5386dddc 2023-07-13 06:00:24,638 INFO [StoreOpener-b06e3484b8c05bbdc046fc6b5386dddc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region b06e3484b8c05bbdc046fc6b5386dddc 2023-07-13 06:00:24,640 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=27 2023-07-13 06:00:24,640 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=27, state=SUCCESS; OpenRegionProcedure 71f62f4fe1b8de992835ee80c072a8c8, server=jenkins-hbase20.apache.org,35187,1689227983701 in 217 msec 2023-07-13 06:00:24,640 DEBUG [StoreOpener-b06e3484b8c05bbdc046fc6b5386dddc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/b06e3484b8c05bbdc046fc6b5386dddc/f1 2023-07-13 06:00:24,640 DEBUG [StoreOpener-b06e3484b8c05bbdc046fc6b5386dddc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/b06e3484b8c05bbdc046fc6b5386dddc/f1 2023-07-13 06:00:24,640 INFO [StoreOpener-b06e3484b8c05bbdc046fc6b5386dddc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b06e3484b8c05bbdc046fc6b5386dddc columnFamilyName f1 2023-07-13 06:00:24,641 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=71f62f4fe1b8de992835ee80c072a8c8, ASSIGN in 390 msec 2023-07-13 06:00:24,641 INFO [StoreOpener-b06e3484b8c05bbdc046fc6b5386dddc-1] regionserver.HStore(310): Store=b06e3484b8c05bbdc046fc6b5386dddc/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-13 06:00:24,642 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/b06e3484b8c05bbdc046fc6b5386dddc 2023-07-13 06:00:24,642 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/b06e3484b8c05bbdc046fc6b5386dddc 2023-07-13 06:00:24,645 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for b06e3484b8c05bbdc046fc6b5386dddc 2023-07-13 06:00:24,647 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/b06e3484b8c05bbdc046fc6b5386dddc/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-13 06:00:24,647 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened b06e3484b8c05bbdc046fc6b5386dddc; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71122398, jitterRate=0.05980631709098816}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-13 06:00:24,647 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for b06e3484b8c05bbdc046fc6b5386dddc: 2023-07-13 06:00:24,648 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc., pid=38, masterSystemTime=1689228024567 2023-07-13 06:00:24,649 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc. 2023-07-13 06:00:24,649 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc. 2023-07-13 06:00:24,650 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d. 2023-07-13 06:00:24,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 0d24aa4917843c73c4cba790560f2b8d, NAME => 'testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d.', STARTKEY => '4', ENDKEY => '5'} 2023-07-13 06:00:24,650 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=b06e3484b8c05bbdc046fc6b5386dddc, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:24,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testTablesWithSnapshots4 0d24aa4917843c73c4cba790560f2b8d 2023-07-13 06:00:24,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 06:00:24,650 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc.","families":{"info":[{"qualifier":"regioninfo","vlen":60,"tag":[],"timestamp":"1689228024650"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689228024650"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689228024650"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689228024650"}]},"ts":"1689228024650"} 2023-07-13 06:00:24,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 0d24aa4917843c73c4cba790560f2b8d 2023-07-13 06:00:24,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 0d24aa4917843c73c4cba790560f2b8d 2023-07-13 06:00:24,652 INFO [StoreOpener-0d24aa4917843c73c4cba790560f2b8d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region 0d24aa4917843c73c4cba790560f2b8d 2023-07-13 06:00:24,653 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=28 2023-07-13 06:00:24,654 DEBUG [StoreOpener-0d24aa4917843c73c4cba790560f2b8d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/0d24aa4917843c73c4cba790560f2b8d/f1 2023-07-13 06:00:24,654 DEBUG [StoreOpener-0d24aa4917843c73c4cba790560f2b8d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/0d24aa4917843c73c4cba790560f2b8d/f1 2023-07-13 06:00:24,654 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=28, state=SUCCESS; OpenRegionProcedure b06e3484b8c05bbdc046fc6b5386dddc, server=jenkins-hbase20.apache.org,35187,1689227983701 in 237 msec 2023-07-13 06:00:24,654 INFO [StoreOpener-0d24aa4917843c73c4cba790560f2b8d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0d24aa4917843c73c4cba790560f2b8d columnFamilyName f1 2023-07-13 06:00:24,655 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=26, state=SUCCESS; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=b06e3484b8c05bbdc046fc6b5386dddc, ASSIGN in 404 msec 2023-07-13 06:00:24,655 INFO [StoreOpener-0d24aa4917843c73c4cba790560f2b8d-1] regionserver.HStore(310): Store=0d24aa4917843c73c4cba790560f2b8d/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-13 06:00:24,656 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/0d24aa4917843c73c4cba790560f2b8d 2023-07-13 06:00:24,656 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/0d24aa4917843c73c4cba790560f2b8d 2023-07-13 06:00:24,658 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 0d24aa4917843c73c4cba790560f2b8d 2023-07-13 06:00:24,659 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/0d24aa4917843c73c4cba790560f2b8d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-13 06:00:24,660 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 0d24aa4917843c73c4cba790560f2b8d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63216790, jitterRate=-0.05799642205238342}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-13 06:00:24,660 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 0d24aa4917843c73c4cba790560f2b8d: 2023-07-13 06:00:24,660 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d., pid=39, masterSystemTime=1689228024567 2023-07-13 06:00:24,662 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d. 2023-07-13 06:00:24,662 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d. 2023-07-13 06:00:24,662 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04. 2023-07-13 06:00:24,662 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e16a1a1ea32dc6c137274851c98ece04, NAME => 'testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04.', STARTKEY => '5', ENDKEY => '6'} 2023-07-13 06:00:24,662 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=0d24aa4917843c73c4cba790560f2b8d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:24,662 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testTablesWithSnapshots4 e16a1a1ea32dc6c137274851c98ece04 2023-07-13 06:00:24,662 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 06:00:24,662 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d.","families":{"info":[{"qualifier":"regioninfo","vlen":60,"tag":[],"timestamp":"1689228024662"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689228024662"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689228024662"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689228024662"}]},"ts":"1689228024662"} 2023-07-13 06:00:24,662 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for e16a1a1ea32dc6c137274851c98ece04 2023-07-13 06:00:24,662 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for e16a1a1ea32dc6c137274851c98ece04 2023-07-13 06:00:24,663 INFO [StoreOpener-e16a1a1ea32dc6c137274851c98ece04-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region e16a1a1ea32dc6c137274851c98ece04 2023-07-13 06:00:24,665 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=32 2023-07-13 06:00:24,665 DEBUG [StoreOpener-e16a1a1ea32dc6c137274851c98ece04-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/e16a1a1ea32dc6c137274851c98ece04/f1 2023-07-13 06:00:24,665 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=32, state=SUCCESS; OpenRegionProcedure 0d24aa4917843c73c4cba790560f2b8d, server=jenkins-hbase20.apache.org,35187,1689227983701 in 248 msec 2023-07-13 06:00:24,665 DEBUG [StoreOpener-e16a1a1ea32dc6c137274851c98ece04-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/e16a1a1ea32dc6c137274851c98ece04/f1 2023-07-13 06:00:24,666 INFO [StoreOpener-e16a1a1ea32dc6c137274851c98ece04-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e16a1a1ea32dc6c137274851c98ece04 columnFamilyName f1 2023-07-13 06:00:24,666 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=26, state=SUCCESS; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=0d24aa4917843c73c4cba790560f2b8d, ASSIGN in 415 msec 2023-07-13 06:00:24,666 INFO [StoreOpener-e16a1a1ea32dc6c137274851c98ece04-1] regionserver.HStore(310): Store=e16a1a1ea32dc6c137274851c98ece04/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-13 06:00:24,667 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/e16a1a1ea32dc6c137274851c98ece04 2023-07-13 06:00:24,668 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/e16a1a1ea32dc6c137274851c98ece04 2023-07-13 06:00:24,671 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for e16a1a1ea32dc6c137274851c98ece04 2023-07-13 06:00:24,673 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/e16a1a1ea32dc6c137274851c98ece04/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-13 06:00:24,673 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened e16a1a1ea32dc6c137274851c98ece04; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68871059, jitterRate=0.02625875174999237}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-13 06:00:24,673 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for e16a1a1ea32dc6c137274851c98ece04: 2023-07-13 06:00:24,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04., pid=46, masterSystemTime=1689228024567 2023-07-13 06:00:24,676 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04. 2023-07-13 06:00:24,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04. 2023-07-13 06:00:24,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3. 2023-07-13 06:00:24,676 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 186edc75693ac04a787ff0d8059708f3, NAME => 'testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3.', STARTKEY => '7', ENDKEY => '9'} 2023-07-13 06:00:24,676 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=33 updating hbase:meta row=e16a1a1ea32dc6c137274851c98ece04, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:24,676 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testTablesWithSnapshots4 186edc75693ac04a787ff0d8059708f3 2023-07-13 06:00:24,676 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 06:00:24,676 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04.","families":{"info":[{"qualifier":"regioninfo","vlen":60,"tag":[],"timestamp":"1689228024676"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689228024676"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689228024676"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689228024676"}]},"ts":"1689228024676"} 2023-07-13 06:00:24,676 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 186edc75693ac04a787ff0d8059708f3 2023-07-13 06:00:24,676 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 186edc75693ac04a787ff0d8059708f3 2023-07-13 06:00:24,678 INFO [StoreOpener-186edc75693ac04a787ff0d8059708f3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region 186edc75693ac04a787ff0d8059708f3 2023-07-13 06:00:24,679 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=33 2023-07-13 06:00:24,679 DEBUG [StoreOpener-186edc75693ac04a787ff0d8059708f3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/186edc75693ac04a787ff0d8059708f3/f1 2023-07-13 06:00:24,679 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=33, state=SUCCESS; OpenRegionProcedure e16a1a1ea32dc6c137274851c98ece04, server=jenkins-hbase20.apache.org,35187,1689227983701 in 252 msec 2023-07-13 06:00:24,679 DEBUG [StoreOpener-186edc75693ac04a787ff0d8059708f3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/186edc75693ac04a787ff0d8059708f3/f1 2023-07-13 06:00:24,680 INFO [StoreOpener-186edc75693ac04a787ff0d8059708f3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 186edc75693ac04a787ff0d8059708f3 columnFamilyName f1 2023-07-13 06:00:24,680 INFO [StoreOpener-186edc75693ac04a787ff0d8059708f3-1] regionserver.HStore(310): Store=186edc75693ac04a787ff0d8059708f3/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-13 06:00:24,680 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=26, state=SUCCESS; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=e16a1a1ea32dc6c137274851c98ece04, ASSIGN in 429 msec 2023-07-13 06:00:24,681 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/186edc75693ac04a787ff0d8059708f3 2023-07-13 06:00:24,681 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/186edc75693ac04a787ff0d8059708f3 2023-07-13 06:00:24,684 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 186edc75693ac04a787ff0d8059708f3 2023-07-13 06:00:24,686 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/186edc75693ac04a787ff0d8059708f3/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-13 06:00:24,687 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 186edc75693ac04a787ff0d8059708f3; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65502115, jitterRate=-0.023942425847053528}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-13 06:00:24,687 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 186edc75693ac04a787ff0d8059708f3: 2023-07-13 06:00:24,687 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3., pid=37, masterSystemTime=1689228024567 2023-07-13 06:00:24,688 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3. 2023-07-13 06:00:24,688 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3. 2023-07-13 06:00:24,688 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093. 2023-07-13 06:00:24,689 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ea36c09b963764399342ace600417093, NAME => 'testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093.', STARTKEY => '2', ENDKEY => '3'} 2023-07-13 06:00:24,689 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=35 updating hbase:meta row=186edc75693ac04a787ff0d8059708f3, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:24,689 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3.","families":{"info":[{"qualifier":"regioninfo","vlen":60,"tag":[],"timestamp":"1689228024689"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689228024689"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689228024689"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689228024689"}]},"ts":"1689228024689"} 2023-07-13 06:00:24,689 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testTablesWithSnapshots4 ea36c09b963764399342ace600417093 2023-07-13 06:00:24,689 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 06:00:24,689 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for ea36c09b963764399342ace600417093 2023-07-13 06:00:24,689 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for ea36c09b963764399342ace600417093 2023-07-13 06:00:24,690 INFO [StoreOpener-ea36c09b963764399342ace600417093-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region ea36c09b963764399342ace600417093 2023-07-13 06:00:24,691 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=35 2023-07-13 06:00:24,691 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; OpenRegionProcedure 186edc75693ac04a787ff0d8059708f3, server=jenkins-hbase20.apache.org,35187,1689227983701 in 275 msec 2023-07-13 06:00:24,692 DEBUG [StoreOpener-ea36c09b963764399342ace600417093-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/ea36c09b963764399342ace600417093/f1 2023-07-13 06:00:24,692 DEBUG [StoreOpener-ea36c09b963764399342ace600417093-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/ea36c09b963764399342ace600417093/f1 2023-07-13 06:00:24,692 INFO [StoreOpener-ea36c09b963764399342ace600417093-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ea36c09b963764399342ace600417093 columnFamilyName f1 2023-07-13 06:00:24,692 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=35, ppid=26, state=SUCCESS; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=186edc75693ac04a787ff0d8059708f3, ASSIGN in 441 msec 2023-07-13 06:00:24,692 INFO [StoreOpener-ea36c09b963764399342ace600417093-1] regionserver.HStore(310): Store=ea36c09b963764399342ace600417093/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-13 06:00:24,693 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/ea36c09b963764399342ace600417093 2023-07-13 06:00:24,693 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/ea36c09b963764399342ace600417093 2023-07-13 06:00:24,696 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for ea36c09b963764399342ace600417093 2023-07-13 06:00:24,697 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/ea36c09b963764399342ace600417093/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-13 06:00:24,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened ea36c09b963764399342ace600417093; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61824605, jitterRate=-0.07874159514904022}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-13 06:00:24,698 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for ea36c09b963764399342ace600417093: 2023-07-13 06:00:24,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093., pid=40, masterSystemTime=1689228024567 2023-07-13 06:00:24,700 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093. 2023-07-13 06:00:24,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093. 2023-07-13 06:00:24,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834. 2023-07-13 06:00:24,700 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6817724a8c0ba54dffa0384c0e1b4834, NAME => 'testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834.', STARTKEY => '9', ENDKEY => ''} 2023-07-13 06:00:24,700 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=ea36c09b963764399342ace600417093, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:24,701 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testTablesWithSnapshots4 6817724a8c0ba54dffa0384c0e1b4834 2023-07-13 06:00:24,701 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093.","families":{"info":[{"qualifier":"regioninfo","vlen":60,"tag":[],"timestamp":"1689228024700"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689228024700"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689228024700"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689228024700"}]},"ts":"1689228024700"} 2023-07-13 06:00:24,701 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 06:00:24,701 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 6817724a8c0ba54dffa0384c0e1b4834 2023-07-13 06:00:24,701 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 6817724a8c0ba54dffa0384c0e1b4834 2023-07-13 06:00:24,702 INFO [StoreOpener-6817724a8c0ba54dffa0384c0e1b4834-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region 6817724a8c0ba54dffa0384c0e1b4834 2023-07-13 06:00:24,704 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=40, resume processing ppid=30 2023-07-13 06:00:24,704 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=40, ppid=30, state=SUCCESS; OpenRegionProcedure ea36c09b963764399342ace600417093, server=jenkins-hbase20.apache.org,35187,1689227983701 in 283 msec 2023-07-13 06:00:24,704 DEBUG [StoreOpener-6817724a8c0ba54dffa0384c0e1b4834-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/6817724a8c0ba54dffa0384c0e1b4834/f1 2023-07-13 06:00:24,704 DEBUG [StoreOpener-6817724a8c0ba54dffa0384c0e1b4834-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/6817724a8c0ba54dffa0384c0e1b4834/f1 2023-07-13 06:00:24,705 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=26, state=SUCCESS; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=ea36c09b963764399342ace600417093, ASSIGN in 454 msec 2023-07-13 06:00:24,705 INFO [StoreOpener-6817724a8c0ba54dffa0384c0e1b4834-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6817724a8c0ba54dffa0384c0e1b4834 columnFamilyName f1 2023-07-13 06:00:24,706 INFO [StoreOpener-6817724a8c0ba54dffa0384c0e1b4834-1] regionserver.HStore(310): Store=6817724a8c0ba54dffa0384c0e1b4834/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-13 06:00:24,706 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/6817724a8c0ba54dffa0384c0e1b4834 2023-07-13 06:00:24,707 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/6817724a8c0ba54dffa0384c0e1b4834 2023-07-13 06:00:24,709 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 6817724a8c0ba54dffa0384c0e1b4834 2023-07-13 06:00:24,711 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/6817724a8c0ba54dffa0384c0e1b4834/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-13 06:00:24,712 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 6817724a8c0ba54dffa0384c0e1b4834; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71532072, jitterRate=0.0659109354019165}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-13 06:00:24,712 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 6817724a8c0ba54dffa0384c0e1b4834: 2023-07-13 06:00:24,712 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834., pid=45, masterSystemTime=1689228024567 2023-07-13 06:00:24,714 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834. 2023-07-13 06:00:24,714 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834. 2023-07-13 06:00:24,714 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=6817724a8c0ba54dffa0384c0e1b4834, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:24,714 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834.","families":{"info":[{"qualifier":"regioninfo","vlen":59,"tag":[],"timestamp":"1689228024714"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689228024714"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689228024714"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689228024714"}]},"ts":"1689228024714"} 2023-07-13 06:00:24,717 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=36 2023-07-13 06:00:24,717 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=36, state=SUCCESS; OpenRegionProcedure 6817724a8c0ba54dffa0384c0e1b4834, server=jenkins-hbase20.apache.org,35187,1689227983701 in 291 msec 2023-07-13 06:00:24,718 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=26 2023-07-13 06:00:24,718 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=26, state=SUCCESS; TransitRegionStateProcedure table=testTablesWithSnapshots4, region=6817724a8c0ba54dffa0384c0e1b4834, ASSIGN in 467 msec 2023-07-13 06:00:24,719 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=26, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testTablesWithSnapshots4 execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-13 06:00:24,719 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testTablesWithSnapshots4","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689228024719"}]},"ts":"1689228024719"} 2023-07-13 06:00:24,720 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testTablesWithSnapshots4, state=ENABLED in hbase:meta 2023-07-13 06:00:24,722 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=26, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testTablesWithSnapshots4 execute state=CREATE_TABLE_POST_OPERATION 2023-07-13 06:00:24,723 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=26, state=SUCCESS; CreateTableProcedure table=testTablesWithSnapshots4 in 2.5820 sec 2023-07-13 06:00:25,238 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x6738350e to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:25,240 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4d744f24, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:25,242 DEBUG [hconnection-0x4e0a9c16-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:25,244 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:56480, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:26,256 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1230): Checking to see if procedure is done pid=26 2023-07-13 06:00:26,257 INFO [Listener at localhost.localdomain/45717] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testTablesWithSnapshots4, procId: 26 completed 2023-07-13 06:00:26,263 INFO [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots(310): Writing first data set 2023-07-13 06:00:26,281 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x019b5cb2 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:26,284 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@77fbc408, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:26,291 DEBUG [hconnection-0x49ad2abb-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:26,296 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:56484, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:26,387 DEBUG [master/jenkins-hbase20:0.Chore.1] quotas.QuotaObserverChore(336): testTablesWithSnapshots4 moved into observance of table space quota. 2023-07-13 06:00:26,620 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests(539): Data was written to HBase 2023-07-13 06:00:26,624 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-13 06:00:26,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] procedure.ProcedureCoordinator(165): Submitting procedure testTablesWithSnapshots4 2023-07-13 06:00:26,626 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testTablesWithSnapshots4' 2023-07-13 06:00:26,626 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-13 06:00:26,627 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testTablesWithSnapshots4' starting 'acquire' 2023-07-13 06:00:26,627 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testTablesWithSnapshots4', kicking off acquire phase on members. 2023-07-13 06:00:26,627 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testTablesWithSnapshots4 2023-07-13 06:00:26,628 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testTablesWithSnapshots4 2023-07-13 06:00:26,628 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testTablesWithSnapshots4/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:26,628 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-13 06:00:26,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-13 06:00:26,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-13 06:00:26,629 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testTablesWithSnapshots4/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:26,629 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-13 06:00:26,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testTablesWithSnapshots4 2023-07-13 06:00:26,629 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testTablesWithSnapshots4 2023-07-13 06:00:26,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-13 06:00:26,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testTablesWithSnapshots4 2023-07-13 06:00:26,629 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testTablesWithSnapshots4 2023-07-13 06:00:26,630 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testTablesWithSnapshots4 2023-07-13 06:00:26,630 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testTablesWithSnapshots4' with timeout 60000ms 2023-07-13 06:00:26,630 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-13 06:00:26,632 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testTablesWithSnapshots4' starting 'acquire' stage 2023-07-13 06:00:26,632 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37. 2023-07-13 06:00:26,632 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37. started... 2023-07-13 06:00:26,632 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8. 2023-07-13 06:00:26,632 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834. 2023-07-13 06:00:26,632 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8. started... 2023-07-13 06:00:26,632 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 10 regions 2023-07-13 06:00:26,633 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834. started... 2023-07-13 06:00:26,633 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2994958daca2f80926fd1799582c2b37 1/1 column families, dataSize=113.60 KB heapSize=144.39 KB 2023-07-13 06:00:26,633 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 71f62f4fe1b8de992835ee80c072a8c8: 2023-07-13 06:00:26,634 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 6817724a8c0ba54dffa0384c0e1b4834 1/1 column families, dataSize=113.33 KB heapSize=144.04 KB 2023-07-13 06:00:26,633 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-13 06:00:26,634 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8. 2023-07-13 06:00:26,634 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/10 local region flush tasks. 2023-07-13 06:00:26,634 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69. 2023-07-13 06:00:26,634 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69. started... 2023-07-13 06:00:26,635 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 41c84bc63b9629db7bd4a197ebc93b69 1/1 column families, dataSize=113.60 KB heapSize=144.39 KB 2023-07-13 06:00:27,057 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=113.60 KB at sequenceid=85 (bloomFilter=true), to=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/2994958daca2f80926fd1799582c2b37/.tmp/f1/3e409f745f004dcab1da0ad081befa99 2023-07-13 06:00:27,057 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=113.33 KB at sequenceid=85 (bloomFilter=true), to=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/6817724a8c0ba54dffa0384c0e1b4834/.tmp/f1/66ebca1e385b4bb6a4845c563267004a 2023-07-13 06:00:27,066 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=113.60 KB at sequenceid=85 (bloomFilter=true), to=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/41c84bc63b9629db7bd4a197ebc93b69/.tmp/f1/08b41d3cfcac4cf3af01961fbb311929 2023-07-13 06:00:27,069 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/2994958daca2f80926fd1799582c2b37/.tmp/f1/3e409f745f004dcab1da0ad081befa99 as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/2994958daca2f80926fd1799582c2b37/f1/3e409f745f004dcab1da0ad081befa99 2023-07-13 06:00:27,069 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/6817724a8c0ba54dffa0384c0e1b4834/.tmp/f1/66ebca1e385b4bb6a4845c563267004a as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/6817724a8c0ba54dffa0384c0e1b4834/f1/66ebca1e385b4bb6a4845c563267004a 2023-07-13 06:00:27,071 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/41c84bc63b9629db7bd4a197ebc93b69/.tmp/f1/08b41d3cfcac4cf3af01961fbb311929 as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/41c84bc63b9629db7bd4a197ebc93b69/f1/08b41d3cfcac4cf3af01961fbb311929 2023-07-13 06:00:27,073 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/2994958daca2f80926fd1799582c2b37/f1/3e409f745f004dcab1da0ad081befa99, entries=410, sequenceid=85, filesize=119.3 K 2023-07-13 06:00:27,073 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/6817724a8c0ba54dffa0384c0e1b4834/f1/66ebca1e385b4bb6a4845c563267004a, entries=409, sequenceid=85, filesize=119.0 K 2023-07-13 06:00:27,074 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~113.33 KB/116045, heapSize ~144.02 KB/147480, currentSize=0 B/0 for 6817724a8c0ba54dffa0384c0e1b4834 in 440ms, sequenceid=85, compaction requested=false 2023-07-13 06:00:27,074 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~113.60 KB/116329, heapSize ~144.38 KB/147840, currentSize=0 B/0 for 2994958daca2f80926fd1799582c2b37 in 441ms, sequenceid=85, compaction requested=false 2023-07-13 06:00:27,075 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testTablesWithSnapshots4' 2023-07-13 06:00:27,075 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testTablesWithSnapshots4' 2023-07-13 06:00:27,076 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 6817724a8c0ba54dffa0384c0e1b4834: 2023-07-13 06:00:27,076 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2994958daca2f80926fd1799582c2b37: 2023-07-13 06:00:27,076 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834. 2023-07-13 06:00:27,076 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37. 2023-07-13 06:00:27,076 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/10 local region flush tasks. 2023-07-13 06:00:27,076 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/10 local region flush tasks. 2023-07-13 06:00:27,076 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3. 2023-07-13 06:00:27,076 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d. 2023-07-13 06:00:27,076 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3. started... 2023-07-13 06:00:27,076 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d. started... 2023-07-13 06:00:27,077 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 186edc75693ac04a787ff0d8059708f3 1/1 column families, dataSize=226.65 KB heapSize=287.83 KB 2023-07-13 06:00:27,077 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 0d24aa4917843c73c4cba790560f2b8d 1/1 column families, dataSize=113.60 KB heapSize=144.39 KB 2023-07-13 06:00:27,079 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/41c84bc63b9629db7bd4a197ebc93b69/f1/08b41d3cfcac4cf3af01961fbb311929, entries=410, sequenceid=85, filesize=119.3 K 2023-07-13 06:00:27,079 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~113.60 KB/116329, heapSize ~144.38 KB/147840, currentSize=0 B/0 for 41c84bc63b9629db7bd4a197ebc93b69 in 444ms, sequenceid=85, compaction requested=false 2023-07-13 06:00:27,079 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 41c84bc63b9629db7bd4a197ebc93b69: 2023-07-13 06:00:27,079 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69. 2023-07-13 06:00:27,080 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093. 2023-07-13 06:00:27,080 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 4/10 local region flush tasks. 2023-07-13 06:00:27,080 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093. started... 2023-07-13 06:00:27,081 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing ea36c09b963764399342ace600417093 1/1 column families, dataSize=113.60 KB heapSize=144.39 KB 2023-07-13 06:00:27,096 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=113.60 KB at sequenceid=85 (bloomFilter=true), to=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/0d24aa4917843c73c4cba790560f2b8d/.tmp/f1/af2f0b7204f343ee92c1eb2435b38351 2023-07-13 06:00:27,096 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=113.60 KB at sequenceid=85 (bloomFilter=true), to=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/ea36c09b963764399342ace600417093/.tmp/f1/3a683e9afd184ad1ad1227e6d2337cd0 2023-07-13 06:00:27,102 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/0d24aa4917843c73c4cba790560f2b8d/.tmp/f1/af2f0b7204f343ee92c1eb2435b38351 as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/0d24aa4917843c73c4cba790560f2b8d/f1/af2f0b7204f343ee92c1eb2435b38351 2023-07-13 06:00:27,102 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/ea36c09b963764399342ace600417093/.tmp/f1/3a683e9afd184ad1ad1227e6d2337cd0 as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/ea36c09b963764399342ace600417093/f1/3a683e9afd184ad1ad1227e6d2337cd0 2023-07-13 06:00:27,107 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/0d24aa4917843c73c4cba790560f2b8d/f1/af2f0b7204f343ee92c1eb2435b38351, entries=410, sequenceid=85, filesize=119.3 K 2023-07-13 06:00:27,108 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/ea36c09b963764399342ace600417093/f1/3a683e9afd184ad1ad1227e6d2337cd0, entries=410, sequenceid=85, filesize=119.3 K 2023-07-13 06:00:27,109 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~113.60 KB/116329, heapSize ~144.38 KB/147840, currentSize=0 B/0 for 0d24aa4917843c73c4cba790560f2b8d in 31ms, sequenceid=85, compaction requested=false 2023-07-13 06:00:27,109 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 0d24aa4917843c73c4cba790560f2b8d: 2023-07-13 06:00:27,109 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~113.60 KB/116329, heapSize ~144.38 KB/147840, currentSize=0 B/0 for ea36c09b963764399342ace600417093 in 28ms, sequenceid=85, compaction requested=false 2023-07-13 06:00:27,109 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d. 2023-07-13 06:00:27,109 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for ea36c09b963764399342ace600417093: 2023-07-13 06:00:27,109 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 5/10 local region flush tasks. 2023-07-13 06:00:27,109 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093. 2023-07-13 06:00:27,109 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc. 2023-07-13 06:00:27,109 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 6/10 local region flush tasks. 2023-07-13 06:00:27,109 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc. 2023-07-13 06:00:27,109 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc. started... 2023-07-13 06:00:27,109 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc. started... 2023-07-13 06:00:27,110 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 87846b634b85323bd205e28cff8a7ccc 1/1 column families, dataSize=113.33 KB heapSize=144.04 KB 2023-07-13 06:00:27,110 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing b06e3484b8c05bbdc046fc6b5386dddc 1/1 column families, dataSize=113.60 KB heapSize=144.39 KB 2023-07-13 06:00:27,226 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x4995f9b7 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:27,232 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@21c8ebd6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:27,236 DEBUG [hconnection-0x25cb8c33-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:27,238 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:56496, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:27,494 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=226.65 KB at sequenceid=85 (bloomFilter=true), to=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/186edc75693ac04a787ff0d8059708f3/.tmp/f1/78ea895f487c47b886a4dd0565aa64d5 2023-07-13 06:00:27,507 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/186edc75693ac04a787ff0d8059708f3/.tmp/f1/78ea895f487c47b886a4dd0565aa64d5 as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/186edc75693ac04a787ff0d8059708f3/f1/78ea895f487c47b886a4dd0565aa64d5 2023-07-13 06:00:27,514 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/186edc75693ac04a787ff0d8059708f3/f1/78ea895f487c47b886a4dd0565aa64d5, entries=818, sequenceid=85, filesize=233.4 K 2023-07-13 06:00:27,515 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~226.65 KB/232090, heapSize ~287.81 KB/294720, currentSize=0 B/0 for 186edc75693ac04a787ff0d8059708f3 in 439ms, sequenceid=85, compaction requested=false 2023-07-13 06:00:27,515 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 186edc75693ac04a787ff0d8059708f3: 2023-07-13 06:00:27,515 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3. 2023-07-13 06:00:27,515 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04. 2023-07-13 06:00:27,515 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 7/10 local region flush tasks. 2023-07-13 06:00:27,515 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04. started... 2023-07-13 06:00:27,516 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e16a1a1ea32dc6c137274851c98ece04 1/1 column families, dataSize=113.60 KB heapSize=144.39 KB 2023-07-13 06:00:27,526 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=113.60 KB at sequenceid=85 (bloomFilter=true), to=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/b06e3484b8c05bbdc046fc6b5386dddc/.tmp/f1/77315930387547fcb82700980d49b594 2023-07-13 06:00:27,532 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/b06e3484b8c05bbdc046fc6b5386dddc/.tmp/f1/77315930387547fcb82700980d49b594 as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/b06e3484b8c05bbdc046fc6b5386dddc/f1/77315930387547fcb82700980d49b594 2023-07-13 06:00:27,533 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=113.33 KB at sequenceid=85 (bloomFilter=true), to=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/87846b634b85323bd205e28cff8a7ccc/.tmp/f1/df7c07ab0bf2439cb03e7a21a814ddfc 2023-07-13 06:00:27,537 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/b06e3484b8c05bbdc046fc6b5386dddc/f1/77315930387547fcb82700980d49b594, entries=410, sequenceid=85, filesize=119.3 K 2023-07-13 06:00:27,538 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~113.60 KB/116329, heapSize ~144.38 KB/147840, currentSize=0 B/0 for b06e3484b8c05bbdc046fc6b5386dddc in 428ms, sequenceid=85, compaction requested=false 2023-07-13 06:00:27,538 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for b06e3484b8c05bbdc046fc6b5386dddc: 2023-07-13 06:00:27,538 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc. 2023-07-13 06:00:27,538 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 8/10 local region flush tasks. 2023-07-13 06:00:27,539 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/87846b634b85323bd205e28cff8a7ccc/.tmp/f1/df7c07ab0bf2439cb03e7a21a814ddfc as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/87846b634b85323bd205e28cff8a7ccc/f1/df7c07ab0bf2439cb03e7a21a814ddfc 2023-07-13 06:00:27,543 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/87846b634b85323bd205e28cff8a7ccc/f1/df7c07ab0bf2439cb03e7a21a814ddfc, entries=409, sequenceid=85, filesize=119.0 K 2023-07-13 06:00:27,544 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~113.33 KB/116045, heapSize ~144.02 KB/147480, currentSize=0 B/0 for 87846b634b85323bd205e28cff8a7ccc in 434ms, sequenceid=85, compaction requested=false 2023-07-13 06:00:27,544 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 87846b634b85323bd205e28cff8a7ccc: 2023-07-13 06:00:27,544 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc. 2023-07-13 06:00:27,544 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 9/10 local region flush tasks. 2023-07-13 06:00:27,928 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=113.60 KB at sequenceid=85 (bloomFilter=true), to=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/e16a1a1ea32dc6c137274851c98ece04/.tmp/f1/8d1735075c04430f88c4ee7785534ff9 2023-07-13 06:00:27,945 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/e16a1a1ea32dc6c137274851c98ece04/.tmp/f1/8d1735075c04430f88c4ee7785534ff9 as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/e16a1a1ea32dc6c137274851c98ece04/f1/8d1735075c04430f88c4ee7785534ff9 2023-07-13 06:00:27,952 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/e16a1a1ea32dc6c137274851c98ece04/f1/8d1735075c04430f88c4ee7785534ff9, entries=410, sequenceid=85, filesize=119.3 K 2023-07-13 06:00:27,953 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~113.60 KB/116329, heapSize ~144.38 KB/147840, currentSize=0 B/0 for e16a1a1ea32dc6c137274851c98ece04 in 437ms, sequenceid=85, compaction requested=false 2023-07-13 06:00:27,953 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e16a1a1ea32dc6c137274851c98ece04: 2023-07-13 06:00:27,953 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04. 2023-07-13 06:00:27,953 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 10/10 local region flush tasks. 2023-07-13 06:00:27,953 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 10 local region flush tasks. 2023-07-13 06:00:27,953 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:27,953 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testTablesWithSnapshots4' locally acquired 2023-07-13 06:00:27,953 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,35187,1689227983701' joining acquired barrier for procedure (testTablesWithSnapshots4) in zk 2023-07-13 06:00:27,956 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testTablesWithSnapshots4/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:27,956 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testTablesWithSnapshots4 2023-07-13 06:00:27,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testTablesWithSnapshots4/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:27,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:27,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:27,956 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testTablesWithSnapshots4 2023-07-13 06:00:27,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:27,956 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testTablesWithSnapshots4' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-13 06:00:27,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:27,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesWithSnapshots4 2023-07-13 06:00:27,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:27,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:27,959 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,35187,1689227983701' joining acquired barrier for procedure 'testTablesWithSnapshots4' on coordinator 2023-07-13 06:00:27,959 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@24f02001[Count = 0] remaining members to acquire global barrier 2023-07-13 06:00:27,959 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testTablesWithSnapshots4' starting 'in-barrier' execution. 2023-07-13 06:00:27,959 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testTablesWithSnapshots4 2023-07-13 06:00:27,962 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testTablesWithSnapshots4 2023-07-13 06:00:27,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testTablesWithSnapshots4 2023-07-13 06:00:27,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testTablesWithSnapshots4 2023-07-13 06:00:27,962 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testTablesWithSnapshots4' received 'reached' from coordinator. 2023-07-13 06:00:27,962 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testTablesWithSnapshots4' locally completed 2023-07-13 06:00:27,962 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testTablesWithSnapshots4' completed for member 'jenkins-hbase20.apache.org,35187,1689227983701' in zk 2023-07-13 06:00:27,962 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testTablesWithSnapshots4/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:27,962 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-13 06:00:27,964 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testTablesWithSnapshots4/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:27,964 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testTablesWithSnapshots4' has notified controller of completion 2023-07-13 06:00:27,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testTablesWithSnapshots4/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:27,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:27,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:27,964 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-13 06:00:27,964 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testTablesWithSnapshots4' completed. 2023-07-13 06:00:27,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:27,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:27,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesWithSnapshots4 2023-07-13 06:00:27,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:27,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:27,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesWithSnapshots4 2023-07-13 06:00:27,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:27,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testTablesWithSnapshots4' member 'jenkins-hbase20.apache.org,35187,1689227983701': 2023-07-13 06:00:27,968 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,35187,1689227983701' released barrier for procedure'testTablesWithSnapshots4', counting down latch. Waiting for 0 more 2023-07-13 06:00:27,968 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testTablesWithSnapshots4' execution completed 2023-07-13 06:00:27,969 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-13 06:00:27,969 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-13 06:00:27,969 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testTablesWithSnapshots4 2023-07-13 06:00:27,969 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testTablesWithSnapshots4including nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-13 06:00:27,970 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testTablesWithSnapshots4 2023-07-13 06:00:27,970 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testTablesWithSnapshots4 2023-07-13 06:00:27,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testTablesWithSnapshots4 2023-07-13 06:00:27,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:27,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:27,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testTablesWithSnapshots4 2023-07-13 06:00:27,970 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-13 06:00:27,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testTablesWithSnapshots4 2023-07-13 06:00:27,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:27,971 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testTablesWithSnapshots4/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:27,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesWithSnapshots4 2023-07-13 06:00:27,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-13 06:00:27,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-13 06:00:27,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:27,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testTablesWithSnapshots4 2023-07-13 06:00:27,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesWithSnapshots4 2023-07-13 06:00:27,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:27,973 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testTablesWithSnapshots4/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:27,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:27,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesWithSnapshots4 2023-07-13 06:00:27,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:27,976 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testTablesWithSnapshots4/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:27,976 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-13 06:00:27,976 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testTablesWithSnapshots4 2023-07-13 06:00:27,976 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-13 06:00:27,976 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testTablesWithSnapshots4 2023-07-13 06:00:27,976 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-13 06:00:27,976 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testTablesWithSnapshots4/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:27,976 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testTablesWithSnapshots4' 2023-07-13 06:00:27,977 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-13 06:00:27,976 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-13 06:00:27,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-13 06:00:27,977 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testTablesWithSnapshots4 2023-07-13 06:00:27,978 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testTablesWithSnapshots4 2023-07-13 06:00:27,978 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testTablesWithSnapshots4'' to complete. (max 20000 ms per retry) 2023-07-13 06:00:27,978 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-13 06:00:27,978 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testTablesWithSnapshots4 2023-07-13 06:00:27,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-13 06:00:27,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-13 06:00:28,078 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-13 06:00:28,079 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-13 06:00:28,079 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests(542): Data flushed to disk 2023-07-13 06:00:28,079 INFO [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots(314): Creating snapshot 2023-07-13 06:00:28,080 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1583): Client=jenkins//148.251.75.209 snapshot request for:{ ss=testTablesWithSnapshots4snap1 table=testTablesWithSnapshots4 type=FLUSH ttl=0 } 2023-07-13 06:00:28,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1689228028080 (current time:1689228028080). 2023-07-13 06:00:28,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-07-13 06:00:28,080 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] zookeeper.ReadOnlyZKClient(139): Connect 0x007f9282 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:28,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@17659dc8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:28,084 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:28,087 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:56512, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:28,088 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x007f9282 to 127.0.0.1:55963 2023-07-13 06:00:28,088 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:28,088 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-07-13 06:00:28,089 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotManager(648): Table enabled, starting distributed snapshots for { ss=testTablesWithSnapshots4snap1 table=testTablesWithSnapshots4 type=FLUSH ttl=0 } 2023-07-13 06:00:28,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] procedure2.ProcedureExecutor(1029): Stored pid=47, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testTablesWithSnapshots4, type=EXCLUSIVE 2023-07-13 06:00:28,092 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=47, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testTablesWithSnapshots4, type=EXCLUSIVE 2023-07-13 06:00:28,092 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=47, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testTablesWithSnapshots4, type=EXCLUSIVE; timeout=600000, timestamp=1689228628092 2023-07-13 06:00:28,092 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotManager(653): Started snapshot: { ss=testTablesWithSnapshots4snap1 table=testTablesWithSnapshots4 type=FLUSH ttl=0 } 2023-07-13 06:00:28,093 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.TakeSnapshotHandler(174): Running FLUSH table snapshot testTablesWithSnapshots4snap1 C_M_SNAPSHOT_TABLE on table testTablesWithSnapshots4 2023-07-13 06:00:28,093 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=testTablesWithSnapshots4snap1 table=testTablesWithSnapshots4 type=FLUSH ttl=0 }'' to complete. (max 20000 ms per retry) 2023-07-13 06:00:28,094 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-07-13 06:00:28,094 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=47, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testTablesWithSnapshots4, type=EXCLUSIVE 2023-07-13 06:00:28,094 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] procedure2.ProcedureExecutor(1029): Stored pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testTablesWithSnapshots4, type=SHARED 2023-07-13 06:00:28,095 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testTablesWithSnapshots4, type=EXCLUSIVE in 4 msec 2023-07-13 06:00:28,096 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testTablesWithSnapshots4, type=SHARED 2023-07-13 06:00:28,097 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=48, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testTablesWithSnapshots4, type=SHARED; timeout=600000, timestamp=1689228628097 2023-07-13 06:00:28,194 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-07-13 06:00:28,196 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=testTablesWithSnapshots4snap1 table=testTablesWithSnapshots4 type=FLUSH ttl=0 } is done 2023-07-13 06:00:28,196 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotManager(404): Snapshoting '{ ss=testTablesWithSnapshots4snap1 table=testTablesWithSnapshots4 type=FLUSH ttl=0 }' is still in progress! 2023-07-13 06:00:28,197 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-07-13 06:00:28,230 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x3175c2cf to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:28,236 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3c160368, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:28,239 DEBUG [hconnection-0x58213470-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:28,244 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:56522, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:28,308 DEBUG [master/jenkins-hbase20:0.Chore.1] quotas.QuotaObserverChore(336): testTablesWithSnapshots4 moved into observance of table space quota. 2023-07-13 06:00:28,397 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-07-13 06:00:28,398 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=testTablesWithSnapshots4snap1 table=testTablesWithSnapshots4 type=FLUSH ttl=0 } is done 2023-07-13 06:00:28,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotManager(404): Snapshoting '{ ss=testTablesWithSnapshots4snap1 table=testTablesWithSnapshots4 type=FLUSH ttl=0 }' is still in progress! 2023-07-13 06:00:28,399 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-07-13 06:00:28,510 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] procedure.ProcedureCoordinator(165): Submitting procedure testTablesWithSnapshots4snap1 2023-07-13 06:00:28,510 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testTablesWithSnapshots4snap1' 2023-07-13 06:00:28,510 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-07-13 06:00:28,510 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testTablesWithSnapshots4snap1' starting 'acquire' 2023-07-13 06:00:28,510 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testTablesWithSnapshots4snap1', kicking off acquire phase on members. 2023-07-13 06:00:28,510 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/testTablesWithSnapshots4snap1 2023-07-13 06:00:28,510 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/online-snapshot/acquired/testTablesWithSnapshots4snap1 2023-07-13 06:00:28,511 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-07-13 06:00:28,511 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/online-snapshot/acquired/testTablesWithSnapshots4snap1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:28,511 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-07-13 06:00:28,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-13 06:00:28,512 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/acquired/testTablesWithSnapshots4snap1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:28,512 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-13 06:00:28,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/testTablesWithSnapshots4snap1 2023-07-13 06:00:28,512 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/testTablesWithSnapshots4snap1 2023-07-13 06:00:28,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 74 2023-07-13 06:00:28,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/testTablesWithSnapshots4snap1 2023-07-13 06:00:28,512 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot testTablesWithSnapshots4snap1 from table testTablesWithSnapshots4 type FLUSH 2023-07-13 06:00:28,513 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testTablesWithSnapshots4snap1 2023-07-13 06:00:28,513 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testTablesWithSnapshots4snap1' with timeout 300000ms 2023-07-13 06:00:28,513 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-07-13 06:00:28,513 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testTablesWithSnapshots4snap1' starting 'acquire' stage 2023-07-13 06:00:28,513 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testTablesWithSnapshots4snap1' locally acquired 2023-07-13 06:00:28,513 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,35187,1689227983701' joining acquired barrier for procedure (testTablesWithSnapshots4snap1) in zk 2023-07-13 06:00:28,515 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/testTablesWithSnapshots4snap1 2023-07-13 06:00:28,515 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/acquired/testTablesWithSnapshots4snap1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:28,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/acquired/testTablesWithSnapshots4snap1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:28,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:28,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-07-13 06:00:28,515 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/testTablesWithSnapshots4snap1 2023-07-13 06:00:28,515 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testTablesWithSnapshots4snap1' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-13 06:00:28,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:28,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:28,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesWithSnapshots4snap1 2023-07-13 06:00:28,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:28,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:28,518 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,35187,1689227983701' joining acquired barrier for procedure 'testTablesWithSnapshots4snap1' on coordinator 2023-07-13 06:00:28,518 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3d2d8b37[Count = 0] remaining members to acquire global barrier 2023-07-13 06:00:28,518 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testTablesWithSnapshots4snap1' starting 'in-barrier' execution. 2023-07-13 06:00:28,518 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/online-snapshot/reached/testTablesWithSnapshots4snap1 2023-07-13 06:00:28,519 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/testTablesWithSnapshots4snap1 2023-07-13 06:00:28,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/testTablesWithSnapshots4snap1 2023-07-13 06:00:28,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/testTablesWithSnapshots4snap1 2023-07-13 06:00:28,519 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testTablesWithSnapshots4snap1' received 'reached' from coordinator. 2023-07-13 06:00:28,519 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/testTablesWithSnapshots4snap1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:28,519 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-13 06:00:28,520 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37. 2023-07-13 06:00:28,520 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37. started... 2023-07-13 06:00:28,520 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8. 2023-07-13 06:00:28,520 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834. 2023-07-13 06:00:28,520 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834. started... 2023-07-13 06:00:28,520 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] regionserver.HRegion(2446): Flush status journal for 2994958daca2f80926fd1799582c2b37: 2023-07-13 06:00:28,520 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] snapshot.FlushSnapshotSubprocedure(170): Flush Snapshot Tasks submitted for 10 regions 2023-07-13 06:00:28,520 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(301): Waiting for local region snapshots to finish. 2023-07-13 06:00:28,520 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-2] regionserver.HRegion(2446): Flush status journal for 6817724a8c0ba54dffa0384c0e1b4834: 2023-07-13 06:00:28,520 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37.' region-info for snapshot=testTablesWithSnapshots4snap1 2023-07-13 06:00:28,520 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8. started... 2023-07-13 06:00:28,520 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-2] snapshot.SnapshotManifest(238): Storing 'testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834.' region-info for snapshot=testTablesWithSnapshots4snap1 2023-07-13 06:00:28,520 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-07-13 06:00:28,520 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-1] regionserver.HRegion(2446): Flush status journal for 71f62f4fe1b8de992835ee80c072a8c8: 2023-07-13 06:00:28,520 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-2] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-07-13 06:00:28,521 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/2994958daca2f80926fd1799582c2b37/f1/3e409f745f004dcab1da0ad081befa99] hfiles 2023-07-13 06:00:28,521 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/6817724a8c0ba54dffa0384c0e1b4834/f1/66ebca1e385b4bb6a4845c563267004a] hfiles 2023-07-13 06:00:28,521 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-1] snapshot.SnapshotManifest(238): Storing 'testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8.' region-info for snapshot=testTablesWithSnapshots4snap1 2023-07-13 06:00:28,521 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/1): hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/6817724a8c0ba54dffa0384c0e1b4834/f1/66ebca1e385b4bb6a4845c563267004a for snapshot=testTablesWithSnapshots4snap1 2023-07-13 06:00:28,521 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/1): hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/2994958daca2f80926fd1799582c2b37/f1/3e409f745f004dcab1da0ad081befa99 for snapshot=testTablesWithSnapshots4snap1 2023-07-13 06:00:28,521 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-1] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-07-13 06:00:28,521 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [] hfiles 2023-07-13 06:00:28,538 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8. completed. 2023-07-13 06:00:28,538 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8. 2023-07-13 06:00:28,538 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69. 2023-07-13 06:00:28,538 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 1/10 local region snapshots. 2023-07-13 06:00:28,538 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69. started... 2023-07-13 06:00:28,539 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-1] regionserver.HRegion(2446): Flush status journal for 41c84bc63b9629db7bd4a197ebc93b69: 2023-07-13 06:00:28,539 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-1] snapshot.SnapshotManifest(238): Storing 'testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69.' region-info for snapshot=testTablesWithSnapshots4snap1 2023-07-13 06:00:28,539 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-1] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-07-13 06:00:28,539 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/41c84bc63b9629db7bd4a197ebc93b69/f1/08b41d3cfcac4cf3af01961fbb311929] hfiles 2023-07-13 06:00:28,539 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/1): hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/41c84bc63b9629db7bd4a197ebc93b69/f1/08b41d3cfcac4cf3af01961fbb311929 for snapshot=testTablesWithSnapshots4snap1 2023-07-13 06:00:28,699 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-07-13 06:00:28,701 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=testTablesWithSnapshots4snap1 table=testTablesWithSnapshots4 type=FLUSH ttl=0 } is done 2023-07-13 06:00:28,701 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotManager(404): Snapshoting '{ ss=testTablesWithSnapshots4snap1 table=testTablesWithSnapshots4 type=FLUSH ttl=0 }' is still in progress! 2023-07-13 06:00:28,701 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-07-13 06:00:28,939 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834. completed. 2023-07-13 06:00:28,939 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834. 2023-07-13 06:00:28,940 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d. 2023-07-13 06:00:28,940 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 2/10 local region snapshots. 2023-07-13 06:00:28,940 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d. started... 2023-07-13 06:00:28,940 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37. completed. 2023-07-13 06:00:28,941 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37. 2023-07-13 06:00:28,941 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-2] regionserver.HRegion(2446): Flush status journal for 0d24aa4917843c73c4cba790560f2b8d: 2023-07-13 06:00:28,941 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 3/10 local region snapshots. 2023-07-13 06:00:28,941 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3. 2023-07-13 06:00:28,942 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-2] snapshot.SnapshotManifest(238): Storing 'testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d.' region-info for snapshot=testTablesWithSnapshots4snap1 2023-07-13 06:00:28,942 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3. started... 2023-07-13 06:00:28,942 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-2] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-07-13 06:00:28,942 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] regionserver.HRegion(2446): Flush status journal for 186edc75693ac04a787ff0d8059708f3: 2023-07-13 06:00:28,942 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/0d24aa4917843c73c4cba790560f2b8d/f1/af2f0b7204f343ee92c1eb2435b38351] hfiles 2023-07-13 06:00:28,942 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/1): hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/0d24aa4917843c73c4cba790560f2b8d/f1/af2f0b7204f343ee92c1eb2435b38351 for snapshot=testTablesWithSnapshots4snap1 2023-07-13 06:00:28,942 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3.' region-info for snapshot=testTablesWithSnapshots4snap1 2023-07-13 06:00:28,942 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-07-13 06:00:28,942 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/186edc75693ac04a787ff0d8059708f3/f1/78ea895f487c47b886a4dd0565aa64d5] hfiles 2023-07-13 06:00:28,942 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/1): hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/186edc75693ac04a787ff0d8059708f3/f1/78ea895f487c47b886a4dd0565aa64d5 for snapshot=testTablesWithSnapshots4snap1 2023-07-13 06:00:28,948 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69. completed. 2023-07-13 06:00:28,949 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69. 2023-07-13 06:00:28,949 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093. 2023-07-13 06:00:28,949 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 4/10 local region snapshots. 2023-07-13 06:00:28,949 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093. started... 2023-07-13 06:00:28,949 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-1] regionserver.HRegion(2446): Flush status journal for ea36c09b963764399342ace600417093: 2023-07-13 06:00:28,949 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-1] snapshot.SnapshotManifest(238): Storing 'testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093.' region-info for snapshot=testTablesWithSnapshots4snap1 2023-07-13 06:00:28,949 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-1] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-07-13 06:00:28,949 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/ea36c09b963764399342ace600417093/f1/3a683e9afd184ad1ad1227e6d2337cd0] hfiles 2023-07-13 06:00:28,949 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/1): hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/ea36c09b963764399342ace600417093/f1/3a683e9afd184ad1ad1227e6d2337cd0 for snapshot=testTablesWithSnapshots4snap1 2023-07-13 06:00:29,201 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-07-13 06:00:29,202 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=testTablesWithSnapshots4snap1 table=testTablesWithSnapshots4 type=FLUSH ttl=0 } is done 2023-07-13 06:00:29,203 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotManager(404): Snapshoting '{ ss=testTablesWithSnapshots4snap1 table=testTablesWithSnapshots4 type=FLUSH ttl=0 }' is still in progress! 2023-07-13 06:00:29,203 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-07-13 06:00:29,226 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x58620e3a to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:29,230 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@12ac38df, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:29,233 DEBUG [hconnection-0x544c1268-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:29,235 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:41440, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:29,306 DEBUG [master/jenkins-hbase20:0.Chore.1] quotas.QuotaObserverChore(336): testTablesWithSnapshots4 moved into observance of table space quota. 2023-07-13 06:00:29,350 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d. completed. 2023-07-13 06:00:29,350 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d. 2023-07-13 06:00:29,350 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc. 2023-07-13 06:00:29,350 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 5/10 local region snapshots. 2023-07-13 06:00:29,350 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3. completed. 2023-07-13 06:00:29,351 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3. 2023-07-13 06:00:29,351 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc. 2023-07-13 06:00:29,351 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc. started... 2023-07-13 06:00:29,350 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc. started... 2023-07-13 06:00:29,351 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] regionserver.HRegion(2446): Flush status journal for b06e3484b8c05bbdc046fc6b5386dddc: 2023-07-13 06:00:29,351 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 6/10 local region snapshots. 2023-07-13 06:00:29,351 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-2] regionserver.HRegion(2446): Flush status journal for 87846b634b85323bd205e28cff8a7ccc: 2023-07-13 06:00:29,351 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc.' region-info for snapshot=testTablesWithSnapshots4snap1 2023-07-13 06:00:29,351 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-2] snapshot.SnapshotManifest(238): Storing 'testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc.' region-info for snapshot=testTablesWithSnapshots4snap1 2023-07-13 06:00:29,351 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-07-13 06:00:29,351 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-2] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-07-13 06:00:29,351 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/b06e3484b8c05bbdc046fc6b5386dddc/f1/77315930387547fcb82700980d49b594] hfiles 2023-07-13 06:00:29,352 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/87846b634b85323bd205e28cff8a7ccc/f1/df7c07ab0bf2439cb03e7a21a814ddfc] hfiles 2023-07-13 06:00:29,352 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/1): hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/b06e3484b8c05bbdc046fc6b5386dddc/f1/77315930387547fcb82700980d49b594 for snapshot=testTablesWithSnapshots4snap1 2023-07-13 06:00:29,352 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/1): hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/87846b634b85323bd205e28cff8a7ccc/f1/df7c07ab0bf2439cb03e7a21a814ddfc for snapshot=testTablesWithSnapshots4snap1 2023-07-13 06:00:29,356 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093. completed. 2023-07-13 06:00:29,356 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093. 2023-07-13 06:00:29,356 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04. 2023-07-13 06:00:29,356 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 7/10 local region snapshots. 2023-07-13 06:00:29,356 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04. started... 2023-07-13 06:00:29,356 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-1] regionserver.HRegion(2446): Flush status journal for e16a1a1ea32dc6c137274851c98ece04: 2023-07-13 06:00:29,356 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-1] snapshot.SnapshotManifest(238): Storing 'testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04.' region-info for snapshot=testTablesWithSnapshots4snap1 2023-07-13 06:00:29,356 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-1] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-07-13 06:00:29,357 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/e16a1a1ea32dc6c137274851c98ece04/f1/8d1735075c04430f88c4ee7785534ff9] hfiles 2023-07-13 06:00:29,357 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/1): hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/e16a1a1ea32dc6c137274851c98ece04/f1/8d1735075c04430f88c4ee7785534ff9 for snapshot=testTablesWithSnapshots4snap1 2023-07-13 06:00:29,361 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc. completed. 2023-07-13 06:00:29,361 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc. 2023-07-13 06:00:29,361 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 8/10 local region snapshots. 2023-07-13 06:00:29,760 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc. completed. 2023-07-13 06:00:29,760 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc. 2023-07-13 06:00:29,761 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 9/10 local region snapshots. 2023-07-13 06:00:29,766 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04. completed. 2023-07-13 06:00:29,766 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04. 2023-07-13 06:00:29,766 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 10/10 local region snapshots. 2023-07-13 06:00:29,766 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(314): Completed 10 local region snapshots. 2023-07-13 06:00:29,766 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(345): cancelling 0 tasks for snapshot jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:29,766 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testTablesWithSnapshots4snap1' locally completed 2023-07-13 06:00:29,766 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testTablesWithSnapshots4snap1' completed for member 'jenkins-hbase20.apache.org,35187,1689227983701' in zk 2023-07-13 06:00:29,769 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/testTablesWithSnapshots4snap1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:29,769 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testTablesWithSnapshots4snap1' has notified controller of completion 2023-07-13 06:00:29,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/reached/testTablesWithSnapshots4snap1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:29,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:29,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-07-13 06:00:29,769 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-13 06:00:29,770 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testTablesWithSnapshots4snap1' completed. 2023-07-13 06:00:29,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:29,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:29,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesWithSnapshots4snap1 2023-07-13 06:00:29,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:29,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:29,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesWithSnapshots4snap1 2023-07-13 06:00:29,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:29,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testTablesWithSnapshots4snap1' member 'jenkins-hbase20.apache.org,35187,1689227983701': 2023-07-13 06:00:29,774 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,35187,1689227983701' released barrier for procedure'testTablesWithSnapshots4snap1', counting down latch. Waiting for 0 more 2023-07-13 06:00:29,774 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testTablesWithSnapshots4snap1' execution completed 2023-07-13 06:00:29,774 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-13 06:00:29,774 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-13 06:00:29,774 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testTablesWithSnapshots4snap1 2023-07-13 06:00:29,774 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testTablesWithSnapshots4snap1including nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2023-07-13 06:00:29,775 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/testTablesWithSnapshots4snap1 2023-07-13 06:00:29,775 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-07-13 06:00:29,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/testTablesWithSnapshots4snap1 2023-07-13 06:00:29,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/testTablesWithSnapshots4snap1 2023-07-13 06:00:29,776 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/testTablesWithSnapshots4snap1 2023-07-13 06:00:29,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/abort/testTablesWithSnapshots4snap1 2023-07-13 06:00:29,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:29,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-07-13 06:00:29,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:29,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-07-13 06:00:29,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesWithSnapshots4snap1 2023-07-13 06:00:29,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-13 06:00:29,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:29,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesWithSnapshots4snap1 2023-07-13 06:00:29,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/testTablesWithSnapshots4snap1 2023-07-13 06:00:29,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:29,778 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/testTablesWithSnapshots4snap1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:29,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:29,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesWithSnapshots4snap1 2023-07-13 06:00:29,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:29,779 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/testTablesWithSnapshots4snap1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:29,781 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/testTablesWithSnapshots4snap1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:29,781 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-07-13 06:00:29,781 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-07-13 06:00:29,781 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired/testTablesWithSnapshots4snap1 2023-07-13 06:00:29,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-07-13 06:00:29,781 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/testTablesWithSnapshots4snap1 2023-07-13 06:00:29,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-13 06:00:29,782 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-13 06:00:29,782 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.EnabledTableSnapshotHandler(97): Done waiting - online snapshot for testTablesWithSnapshots4snap1 2023-07-13 06:00:29,782 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/testTablesWithSnapshots4snap1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:29,782 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for testTablesWithSnapshots4snap1 2023-07-13 06:00:29,782 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/reached/testTablesWithSnapshots4snap1 2023-07-13 06:00:29,783 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/testTablesWithSnapshots4snap1 2023-07-13 06:00:29,783 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/abort/testTablesWithSnapshots4snap1 2023-07-13 06:00:29,783 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-07-13 06:00:29,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-13 06:00:29,783 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.hbase-snapshot/.tmp/testTablesWithSnapshots4snap1 2023-07-13 06:00:29,851 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.hbase-snapshot/.tmp/testTablesWithSnapshots4snap1 to hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.hbase-snapshot/testTablesWithSnapshots4snap1 2023-07-13 06:00:30,103 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-13 06:00:30,203 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-07-13 06:00:30,204 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=testTablesWithSnapshots4snap1 table=testTablesWithSnapshots4 type=FLUSH ttl=0 } is done 2023-07-13 06:00:30,204 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotManager(404): Snapshoting '{ ss=testTablesWithSnapshots4snap1 table=testTablesWithSnapshots4 type=FLUSH ttl=0 }' is still in progress! 2023-07-13 06:00:30,204 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2428): (#6) Sleeping: 2000ms while waiting for snapshot completion. 2023-07-13 06:00:30,228 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x2e53a26f to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:30,232 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2ff86aed, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:30,233 DEBUG [hconnection-0x28f042e8-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:30,236 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:41456, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:30,674 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.TakeSnapshotHandler(229): Snapshot testTablesWithSnapshots4snap1 of table testTablesWithSnapshots4 completed 2023-07-13 06:00:30,674 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.hbase-snapshot/.tmp/testTablesWithSnapshots4snap1 2023-07-13 06:00:30,674 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.hbase-snapshot/.tmp/testTablesWithSnapshots4snap1 2023-07-13 06:00:30,674 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running FLUSH table snapshot testTablesWithSnapshots4snap1 C_M_SNAPSHOT_TABLE on table testTablesWithSnapshots4 at 1689228028093Consolidate snapshot: testTablesWithSnapshots4snap1 at 1689228029782 (+1689 ms)Loading Region manifests for testTablesWithSnapshots4snap1 at 1689228029783 (+1 ms)Writing data manifest for testTablesWithSnapshots4snap1 at 1689228029822 (+39 ms)Verifying snapshot: testTablesWithSnapshots4snap1 at 1689228029838 (+16 ms)Snapshot testTablesWithSnapshots4snap1 of table testTablesWithSnapshots4 completed at 1689228030674 (+836 ms) 2023-07-13 06:00:30,676 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=48, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testTablesWithSnapshots4, type=SHARED 2023-07-13 06:00:30,677 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testTablesWithSnapshots4, type=SHARED in 2.5830 sec 2023-07-13 06:00:31,299 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x6a257f54 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:31,306 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7067e9fb, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:31,309 DEBUG [hconnection-0x6d12bd8a-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:31,312 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:41468, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:32,205 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-07-13 06:00:32,207 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=testTablesWithSnapshots4snap1 table=testTablesWithSnapshots4 type=FLUSH ttl=0 } is done 2023-07-13 06:00:32,208 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotManager(401): Snapshot '{ ss=testTablesWithSnapshots4snap1 table=testTablesWithSnapshots4 type=FLUSH ttl=0 }' has completed, notifying client. 2023-07-13 06:00:32,208 INFO [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots(317): Writing second data set 2023-07-13 06:00:32,242 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x211cbd21 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:32,245 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5c79a47d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:32,246 DEBUG [hconnection-0x30290538-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:32,247 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:41482, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:32,444 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7a6b06d6 to 127.0.0.1:55963 2023-07-13 06:00:32,444 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:32,445 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2e53a26f to 127.0.0.1:55963 2023-07-13 06:00:32,445 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:32,445 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6a257f54 to 127.0.0.1:55963 2023-07-13 06:00:32,445 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:32,446 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x211cbd21 to 127.0.0.1:55963 2023-07-13 06:00:32,446 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:32,446 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3175c2cf to 127.0.0.1:55963 2023-07-13 06:00:32,446 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:32,446 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x58620e3a to 127.0.0.1:55963 2023-07-13 06:00:32,446 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:32,446 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x019b5cb2 to 127.0.0.1:55963 2023-07-13 06:00:32,446 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:32,446 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4995f9b7 to 127.0.0.1:55963 2023-07-13 06:00:32,446 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:32,447 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5d3c72df to 127.0.0.1:55963 2023-07-13 06:00:32,447 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:32,447 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x49335a62 to 127.0.0.1:55963 2023-07-13 06:00:32,447 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:32,447 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6738350e to 127.0.0.1:55963 2023-07-13 06:00:32,447 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:32,447 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0e7d55b9 to 127.0.0.1:55963 2023-07-13 06:00:32,447 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:32,447 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5b1b9523 to 127.0.0.1:55963 2023-07-13 06:00:32,447 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:32,447 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x66be4eee to 127.0.0.1:55963 2023-07-13 06:00:32,447 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:32,447 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x62bd7799 to 127.0.0.1:55963 2023-07-13 06:00:32,447 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:32,448 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x105895c2 to 127.0.0.1:55963 2023-07-13 06:00:32,448 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:32,448 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6a473778 to 127.0.0.1:55963 2023-07-13 06:00:32,448 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:32,448 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5d5530f7 to 127.0.0.1:55963 2023-07-13 06:00:32,448 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:32,448 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4efa43a3 to 127.0.0.1:55963 2023-07-13 06:00:32,448 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:32,448 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x121da4ef to 127.0.0.1:55963 2023-07-13 06:00:32,448 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:32,448 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1624ff10 to 127.0.0.1:55963 2023-07-13 06:00:32,448 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:32,536 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests(539): Data was written to HBase 2023-07-13 06:00:32,538 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-13 06:00:32,540 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] procedure.ProcedureCoordinator(143): Procedure testTablesWithSnapshots4 was in running list but was completed. Accepting new attempt. 2023-07-13 06:00:32,540 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] procedure.ProcedureCoordinator(165): Submitting procedure testTablesWithSnapshots4 2023-07-13 06:00:32,540 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testTablesWithSnapshots4' 2023-07-13 06:00:32,540 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-13 06:00:32,541 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testTablesWithSnapshots4' starting 'acquire' 2023-07-13 06:00:32,541 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testTablesWithSnapshots4', kicking off acquire phase on members. 2023-07-13 06:00:32,550 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testTablesWithSnapshots4 2023-07-13 06:00:32,550 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testTablesWithSnapshots4 2023-07-13 06:00:32,651 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-13 06:00:32,651 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testTablesWithSnapshots4/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:32,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-13 06:00:32,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-13 06:00:32,652 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testTablesWithSnapshots4/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:32,652 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-13 06:00:32,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testTablesWithSnapshots4 2023-07-13 06:00:32,652 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testTablesWithSnapshots4 2023-07-13 06:00:32,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-13 06:00:32,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testTablesWithSnapshots4 2023-07-13 06:00:32,652 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testTablesWithSnapshots4 2023-07-13 06:00:32,653 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testTablesWithSnapshots4 2023-07-13 06:00:32,653 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testTablesWithSnapshots4' with timeout 60000ms 2023-07-13 06:00:32,653 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-13 06:00:32,653 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testTablesWithSnapshots4' starting 'acquire' stage 2023-07-13 06:00:32,654 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37. 2023-07-13 06:00:32,654 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37. started... 2023-07-13 06:00:32,654 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8. 2023-07-13 06:00:32,654 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8. started... 2023-07-13 06:00:32,654 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 10 regions 2023-07-13 06:00:32,654 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-13 06:00:32,654 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 2994958daca2f80926fd1799582c2b37 1/1 column families, dataSize=113.60 KB heapSize=144.39 KB 2023-07-13 06:00:32,654 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834. 2023-07-13 06:00:32,654 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 71f62f4fe1b8de992835ee80c072a8c8: 2023-07-13 06:00:32,654 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834. started... 2023-07-13 06:00:32,654 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8. 2023-07-13 06:00:32,654 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69. 2023-07-13 06:00:32,654 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/10 local region flush tasks. 2023-07-13 06:00:32,654 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 6817724a8c0ba54dffa0384c0e1b4834 1/1 column families, dataSize=113.33 KB heapSize=144.04 KB 2023-07-13 06:00:32,654 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69. started... 2023-07-13 06:00:32,655 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 41c84bc63b9629db7bd4a197ebc93b69 1/1 column families, dataSize=113.60 KB heapSize=144.39 KB 2023-07-13 06:00:33,066 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=113.60 KB at sequenceid=170 (bloomFilter=true), to=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/2994958daca2f80926fd1799582c2b37/.tmp/f1/e3550829b34146b591d67cb4c3543ecf 2023-07-13 06:00:33,066 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=113.33 KB at sequenceid=170 (bloomFilter=true), to=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/6817724a8c0ba54dffa0384c0e1b4834/.tmp/f1/37d03617d0a3478793a526fe0961f0e0 2023-07-13 06:00:33,070 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=113.60 KB at sequenceid=170 (bloomFilter=true), to=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/41c84bc63b9629db7bd4a197ebc93b69/.tmp/f1/a3afa38e743544329100dabfb01fd8fd 2023-07-13 06:00:33,079 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/2994958daca2f80926fd1799582c2b37/.tmp/f1/e3550829b34146b591d67cb4c3543ecf as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/2994958daca2f80926fd1799582c2b37/f1/e3550829b34146b591d67cb4c3543ecf 2023-07-13 06:00:33,079 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/6817724a8c0ba54dffa0384c0e1b4834/.tmp/f1/37d03617d0a3478793a526fe0961f0e0 as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/6817724a8c0ba54dffa0384c0e1b4834/f1/37d03617d0a3478793a526fe0961f0e0 2023-07-13 06:00:33,081 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/41c84bc63b9629db7bd4a197ebc93b69/.tmp/f1/a3afa38e743544329100dabfb01fd8fd as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/41c84bc63b9629db7bd4a197ebc93b69/f1/a3afa38e743544329100dabfb01fd8fd 2023-07-13 06:00:33,084 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/6817724a8c0ba54dffa0384c0e1b4834/f1/37d03617d0a3478793a526fe0961f0e0, entries=409, sequenceid=170, filesize=119.2 K 2023-07-13 06:00:33,084 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/2994958daca2f80926fd1799582c2b37/f1/e3550829b34146b591d67cb4c3543ecf, entries=410, sequenceid=170, filesize=119.5 K 2023-07-13 06:00:33,085 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~113.60 KB/116329, heapSize ~144.38 KB/147840, currentSize=0 B/0 for 2994958daca2f80926fd1799582c2b37 in 431ms, sequenceid=170, compaction requested=false 2023-07-13 06:00:33,085 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~113.33 KB/116045, heapSize ~144.02 KB/147480, currentSize=0 B/0 for 6817724a8c0ba54dffa0384c0e1b4834 in 431ms, sequenceid=170, compaction requested=false 2023-07-13 06:00:33,085 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2994958daca2f80926fd1799582c2b37: 2023-07-13 06:00:33,085 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 6817724a8c0ba54dffa0384c0e1b4834: 2023-07-13 06:00:33,085 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/41c84bc63b9629db7bd4a197ebc93b69/f1/a3afa38e743544329100dabfb01fd8fd, entries=410, sequenceid=170, filesize=119.5 K 2023-07-13 06:00:33,085 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37. 2023-07-13 06:00:33,085 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834. 2023-07-13 06:00:33,085 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/10 local region flush tasks. 2023-07-13 06:00:33,085 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d. 2023-07-13 06:00:33,086 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d. started... 2023-07-13 06:00:33,086 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/10 local region flush tasks. 2023-07-13 06:00:33,086 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3. 2023-07-13 06:00:33,086 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3. started... 2023-07-13 06:00:33,086 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0d24aa4917843c73c4cba790560f2b8d 1/1 column families, dataSize=113.60 KB heapSize=144.39 KB 2023-07-13 06:00:33,086 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~113.60 KB/116329, heapSize ~144.38 KB/147840, currentSize=0 B/0 for 41c84bc63b9629db7bd4a197ebc93b69 in 432ms, sequenceid=170, compaction requested=false 2023-07-13 06:00:33,086 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 41c84bc63b9629db7bd4a197ebc93b69: 2023-07-13 06:00:33,086 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69. 2023-07-13 06:00:33,086 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093. 2023-07-13 06:00:33,086 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 4/10 local region flush tasks. 2023-07-13 06:00:33,086 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing 186edc75693ac04a787ff0d8059708f3 1/1 column families, dataSize=226.65 KB heapSize=287.83 KB 2023-07-13 06:00:33,086 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093. started... 2023-07-13 06:00:33,086 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing ea36c09b963764399342ace600417093 1/1 column families, dataSize=113.60 KB heapSize=144.39 KB 2023-07-13 06:00:33,261 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x5aa82376 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:33,265 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7f6fafae, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:33,267 DEBUG [hconnection-0x77967eaa-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:33,269 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:41484, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:33,502 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=113.60 KB at sequenceid=170 (bloomFilter=true), to=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/0d24aa4917843c73c4cba790560f2b8d/.tmp/f1/2ccdd294cede4d69b2ddedf7d61ce80e 2023-07-13 06:00:33,503 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=113.60 KB at sequenceid=170 (bloomFilter=true), to=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/ea36c09b963764399342ace600417093/.tmp/f1/5fea5c8d7df14a68aa60c36c5c08bd15 2023-07-13 06:00:33,507 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=226.65 KB at sequenceid=170 (bloomFilter=true), to=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/186edc75693ac04a787ff0d8059708f3/.tmp/f1/6d6410d899e64e10b85493e333f9aa1e 2023-07-13 06:00:33,512 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/0d24aa4917843c73c4cba790560f2b8d/.tmp/f1/2ccdd294cede4d69b2ddedf7d61ce80e as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/0d24aa4917843c73c4cba790560f2b8d/f1/2ccdd294cede4d69b2ddedf7d61ce80e 2023-07-13 06:00:33,514 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/186edc75693ac04a787ff0d8059708f3/.tmp/f1/6d6410d899e64e10b85493e333f9aa1e as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/186edc75693ac04a787ff0d8059708f3/f1/6d6410d899e64e10b85493e333f9aa1e 2023-07-13 06:00:33,515 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/ea36c09b963764399342ace600417093/.tmp/f1/5fea5c8d7df14a68aa60c36c5c08bd15 as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/ea36c09b963764399342ace600417093/f1/5fea5c8d7df14a68aa60c36c5c08bd15 2023-07-13 06:00:33,522 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/ea36c09b963764399342ace600417093/f1/5fea5c8d7df14a68aa60c36c5c08bd15, entries=410, sequenceid=170, filesize=119.5 K 2023-07-13 06:00:33,522 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/186edc75693ac04a787ff0d8059708f3/f1/6d6410d899e64e10b85493e333f9aa1e, entries=818, sequenceid=170, filesize=233.8 K 2023-07-13 06:00:33,522 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/0d24aa4917843c73c4cba790560f2b8d/f1/2ccdd294cede4d69b2ddedf7d61ce80e, entries=410, sequenceid=170, filesize=119.5 K 2023-07-13 06:00:33,523 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~226.65 KB/232090, heapSize ~287.81 KB/294720, currentSize=0 B/0 for 186edc75693ac04a787ff0d8059708f3 in 437ms, sequenceid=170, compaction requested=false 2023-07-13 06:00:33,523 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~113.60 KB/116329, heapSize ~144.38 KB/147840, currentSize=0 B/0 for ea36c09b963764399342ace600417093 in 437ms, sequenceid=170, compaction requested=false 2023-07-13 06:00:33,523 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 186edc75693ac04a787ff0d8059708f3: 2023-07-13 06:00:33,523 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for ea36c09b963764399342ace600417093: 2023-07-13 06:00:33,523 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~113.60 KB/116329, heapSize ~144.38 KB/147840, currentSize=0 B/0 for 0d24aa4917843c73c4cba790560f2b8d in 437ms, sequenceid=170, compaction requested=false 2023-07-13 06:00:33,523 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093. 2023-07-13 06:00:33,523 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3. 2023-07-13 06:00:33,523 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 5/10 local region flush tasks. 2023-07-13 06:00:33,523 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc. 2023-07-13 06:00:33,523 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc. started... 2023-07-13 06:00:33,523 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc. 2023-07-13 06:00:33,523 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0d24aa4917843c73c4cba790560f2b8d: 2023-07-13 06:00:33,523 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.HRegion(2745): Flushing b06e3484b8c05bbdc046fc6b5386dddc 1/1 column families, dataSize=113.60 KB heapSize=144.39 KB 2023-07-13 06:00:33,523 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc. started... 2023-07-13 06:00:33,523 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 6/10 local region flush tasks. 2023-07-13 06:00:33,523 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d. 2023-07-13 06:00:33,523 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.HRegion(2745): Flushing 87846b634b85323bd205e28cff8a7ccc 1/1 column families, dataSize=113.33 KB heapSize=144.04 KB 2023-07-13 06:00:33,523 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 7/10 local region flush tasks. 2023-07-13 06:00:33,523 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04. 2023-07-13 06:00:33,524 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04. started... 2023-07-13 06:00:33,524 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e16a1a1ea32dc6c137274851c98ece04 1/1 column families, dataSize=113.60 KB heapSize=144.39 KB 2023-07-13 06:00:33,936 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=113.33 KB at sequenceid=170 (bloomFilter=true), to=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/87846b634b85323bd205e28cff8a7ccc/.tmp/f1/2f6a2d63486d4b11ab778f53ea0f8c9a 2023-07-13 06:00:33,936 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=113.60 KB at sequenceid=170 (bloomFilter=true), to=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/e16a1a1ea32dc6c137274851c98ece04/.tmp/f1/0c78701355e6406ca279301bab655166 2023-07-13 06:00:33,936 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=113.60 KB at sequenceid=170 (bloomFilter=true), to=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/b06e3484b8c05bbdc046fc6b5386dddc/.tmp/f1/b4f4dc659c2e4211abd5d08f08eacbc9 2023-07-13 06:00:33,946 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/87846b634b85323bd205e28cff8a7ccc/.tmp/f1/2f6a2d63486d4b11ab778f53ea0f8c9a as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/87846b634b85323bd205e28cff8a7ccc/f1/2f6a2d63486d4b11ab778f53ea0f8c9a 2023-07-13 06:00:33,946 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/e16a1a1ea32dc6c137274851c98ece04/.tmp/f1/0c78701355e6406ca279301bab655166 as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/e16a1a1ea32dc6c137274851c98ece04/f1/0c78701355e6406ca279301bab655166 2023-07-13 06:00:33,947 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/b06e3484b8c05bbdc046fc6b5386dddc/.tmp/f1/b4f4dc659c2e4211abd5d08f08eacbc9 as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/b06e3484b8c05bbdc046fc6b5386dddc/f1/b4f4dc659c2e4211abd5d08f08eacbc9 2023-07-13 06:00:33,951 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/b06e3484b8c05bbdc046fc6b5386dddc/f1/b4f4dc659c2e4211abd5d08f08eacbc9, entries=410, sequenceid=170, filesize=119.5 K 2023-07-13 06:00:33,951 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/e16a1a1ea32dc6c137274851c98ece04/f1/0c78701355e6406ca279301bab655166, entries=410, sequenceid=170, filesize=119.5 K 2023-07-13 06:00:33,951 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/87846b634b85323bd205e28cff8a7ccc/f1/2f6a2d63486d4b11ab778f53ea0f8c9a, entries=409, sequenceid=170, filesize=119.2 K 2023-07-13 06:00:33,952 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~113.60 KB/116329, heapSize ~144.38 KB/147840, currentSize=0 B/0 for e16a1a1ea32dc6c137274851c98ece04 in 428ms, sequenceid=170, compaction requested=false 2023-07-13 06:00:33,952 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.HRegion(2948): Finished flush of dataSize ~113.33 KB/116045, heapSize ~144.02 KB/147480, currentSize=0 B/0 for 87846b634b85323bd205e28cff8a7ccc in 429ms, sequenceid=170, compaction requested=false 2023-07-13 06:00:33,952 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.HRegion(2948): Finished flush of dataSize ~113.60 KB/116329, heapSize ~144.38 KB/147840, currentSize=0 B/0 for b06e3484b8c05bbdc046fc6b5386dddc in 429ms, sequenceid=170, compaction requested=false 2023-07-13 06:00:33,952 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 87846b634b85323bd205e28cff8a7ccc: 2023-07-13 06:00:33,952 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e16a1a1ea32dc6c137274851c98ece04: 2023-07-13 06:00:33,952 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc. 2023-07-13 06:00:33,952 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for b06e3484b8c05bbdc046fc6b5386dddc: 2023-07-13 06:00:33,952 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 8/10 local region flush tasks. 2023-07-13 06:00:33,952 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04. 2023-07-13 06:00:33,952 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc. 2023-07-13 06:00:33,952 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 9/10 local region flush tasks. 2023-07-13 06:00:33,952 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 10/10 local region flush tasks. 2023-07-13 06:00:33,952 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 10 local region flush tasks. 2023-07-13 06:00:33,952 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:33,952 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testTablesWithSnapshots4' locally acquired 2023-07-13 06:00:33,952 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,35187,1689227983701' joining acquired barrier for procedure (testTablesWithSnapshots4) in zk 2023-07-13 06:00:33,953 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testTablesWithSnapshots4/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:33,953 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testTablesWithSnapshots4 2023-07-13 06:00:33,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testTablesWithSnapshots4/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:33,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:33,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:33,954 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testTablesWithSnapshots4 2023-07-13 06:00:33,954 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testTablesWithSnapshots4' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-13 06:00:33,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:33,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:33,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesWithSnapshots4 2023-07-13 06:00:33,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:33,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:33,955 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,35187,1689227983701' joining acquired barrier for procedure 'testTablesWithSnapshots4' on coordinator 2023-07-13 06:00:33,955 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1db7fbe8[Count = 0] remaining members to acquire global barrier 2023-07-13 06:00:33,955 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testTablesWithSnapshots4' starting 'in-barrier' execution. 2023-07-13 06:00:33,955 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testTablesWithSnapshots4 2023-07-13 06:00:33,956 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testTablesWithSnapshots4 2023-07-13 06:00:33,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testTablesWithSnapshots4 2023-07-13 06:00:33,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testTablesWithSnapshots4 2023-07-13 06:00:33,956 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testTablesWithSnapshots4' received 'reached' from coordinator. 2023-07-13 06:00:33,956 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testTablesWithSnapshots4' locally completed 2023-07-13 06:00:33,956 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testTablesWithSnapshots4' completed for member 'jenkins-hbase20.apache.org,35187,1689227983701' in zk 2023-07-13 06:00:33,956 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testTablesWithSnapshots4/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:33,956 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-13 06:00:33,957 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testTablesWithSnapshots4' has notified controller of completion 2023-07-13 06:00:33,957 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testTablesWithSnapshots4/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:33,957 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-13 06:00:33,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testTablesWithSnapshots4/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:33,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:33,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:33,957 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testTablesWithSnapshots4' completed. 2023-07-13 06:00:33,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:33,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:33,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesWithSnapshots4 2023-07-13 06:00:33,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:33,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:33,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesWithSnapshots4 2023-07-13 06:00:33,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:33,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testTablesWithSnapshots4' member 'jenkins-hbase20.apache.org,35187,1689227983701': 2023-07-13 06:00:33,960 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,35187,1689227983701' released barrier for procedure'testTablesWithSnapshots4', counting down latch. Waiting for 0 more 2023-07-13 06:00:33,960 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testTablesWithSnapshots4' execution completed 2023-07-13 06:00:33,960 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-13 06:00:33,960 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-13 06:00:33,960 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testTablesWithSnapshots4 2023-07-13 06:00:33,960 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testTablesWithSnapshots4including nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-13 06:00:33,961 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testTablesWithSnapshots4 2023-07-13 06:00:33,961 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-13 06:00:33,961 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testTablesWithSnapshots4 2023-07-13 06:00:33,961 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testTablesWithSnapshots4 2023-07-13 06:00:33,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testTablesWithSnapshots4 2023-07-13 06:00:33,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testTablesWithSnapshots4 2023-07-13 06:00:33,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:33,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:33,961 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testTablesWithSnapshots4/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:33,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-13 06:00:33,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:33,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-13 06:00:33,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesWithSnapshots4 2023-07-13 06:00:33,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testTablesWithSnapshots4 2023-07-13 06:00:33,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:33,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesWithSnapshots4 2023-07-13 06:00:33,962 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testTablesWithSnapshots4/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:33,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:33,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:33,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesWithSnapshots4 2023-07-13 06:00:33,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:33,964 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testTablesWithSnapshots4/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:33,964 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testTablesWithSnapshots4 2023-07-13 06:00:33,964 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-13 06:00:33,964 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testTablesWithSnapshots4 2023-07-13 06:00:33,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-13 06:00:33,964 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testTablesWithSnapshots4 because node does not exist (not an error) 2023-07-13 06:00:33,964 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-13 06:00:33,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-13 06:00:33,964 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testTablesWithSnapshots4' 2023-07-13 06:00:33,964 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-13 06:00:33,964 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-13 06:00:33,964 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testTablesWithSnapshots4/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:33,964 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-13 06:00:33,965 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testTablesWithSnapshots4 2023-07-13 06:00:33,965 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testTablesWithSnapshots4'' to complete. (max 20000 ms per retry) 2023-07-13 06:00:33,965 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testTablesWithSnapshots4 2023-07-13 06:00:33,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-13 06:00:33,965 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testTablesWithSnapshots4 2023-07-13 06:00:33,965 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-13 06:00:34,065 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-13 06:00:34,066 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-13 06:00:34,066 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests(542): Data flushed to disk 2023-07-13 06:00:34,066 INFO [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots(321): Flushing and major compacting table 2023-07-13 06:00:34,069 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-13 06:00:34,072 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] procedure.ProcedureCoordinator(143): Procedure testTablesWithSnapshots4 was in running list but was completed. Accepting new attempt. 2023-07-13 06:00:34,072 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] procedure.ProcedureCoordinator(165): Submitting procedure testTablesWithSnapshots4 2023-07-13 06:00:34,072 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testTablesWithSnapshots4' 2023-07-13 06:00:34,072 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-13 06:00:34,072 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testTablesWithSnapshots4' starting 'acquire' 2023-07-13 06:00:34,072 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testTablesWithSnapshots4', kicking off acquire phase on members. 2023-07-13 06:00:34,073 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testTablesWithSnapshots4 2023-07-13 06:00:34,073 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testTablesWithSnapshots4 2023-07-13 06:00:34,225 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x4769ecd3 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:34,295 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-13 06:00:34,295 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testTablesWithSnapshots4/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:34,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-13 06:00:34,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-13 06:00:34,311 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testTablesWithSnapshots4/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:34,312 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-13 06:00:34,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testTablesWithSnapshots4 2023-07-13 06:00:34,312 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@23d954f1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:34,312 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testTablesWithSnapshots4 2023-07-13 06:00:34,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-13 06:00:34,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testTablesWithSnapshots4 2023-07-13 06:00:34,313 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testTablesWithSnapshots4 2023-07-13 06:00:34,314 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testTablesWithSnapshots4 is still present, removing 2023-07-13 06:00:34,314 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testTablesWithSnapshots4 2023-07-13 06:00:34,314 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testTablesWithSnapshots4' with timeout 60000ms 2023-07-13 06:00:34,314 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-13 06:00:34,315 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testTablesWithSnapshots4' starting 'acquire' stage 2023-07-13 06:00:34,316 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37. 2023-07-13 06:00:34,316 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8. 2023-07-13 06:00:34,316 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834. 2023-07-13 06:00:34,316 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 10 regions 2023-07-13 06:00:34,316 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-13 06:00:34,316 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37. started... 2023-07-13 06:00:34,316 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834. started... 2023-07-13 06:00:34,316 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 2994958daca2f80926fd1799582c2b37: 2023-07-13 06:00:34,317 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 6817724a8c0ba54dffa0384c0e1b4834: 2023-07-13 06:00:34,317 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834. 2023-07-13 06:00:34,316 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8. started... 2023-07-13 06:00:34,317 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/10 local region flush tasks. 2023-07-13 06:00:34,317 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69. 2023-07-13 06:00:34,317 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69. started... 2023-07-13 06:00:34,317 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37. 2023-07-13 06:00:34,316 DEBUG [hconnection-0x224a6648-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:34,317 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d. 2023-07-13 06:00:34,317 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/10 local region flush tasks. 2023-07-13 06:00:34,317 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 41c84bc63b9629db7bd4a197ebc93b69: 2023-07-13 06:00:34,318 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69. 2023-07-13 06:00:34,317 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for 71f62f4fe1b8de992835ee80c072a8c8: 2023-07-13 06:00:34,318 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/10 local region flush tasks. 2023-07-13 06:00:34,318 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3. 2023-07-13 06:00:34,318 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3. started... 2023-07-13 06:00:34,317 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d. started... 2023-07-13 06:00:34,318 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for 186edc75693ac04a787ff0d8059708f3: 2023-07-13 06:00:34,318 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8. 2023-07-13 06:00:34,318 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0d24aa4917843c73c4cba790560f2b8d: 2023-07-13 06:00:34,318 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d. 2023-07-13 06:00:34,319 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc. 2023-07-13 06:00:34,319 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc. started... 2023-07-13 06:00:34,318 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3. 2023-07-13 06:00:34,320 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc. 2023-07-13 06:00:34,320 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc. started... 2023-07-13 06:00:34,318 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 4/10 local region flush tasks. 2023-07-13 06:00:34,318 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093. 2023-07-13 06:00:34,320 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093. started... 2023-07-13 06:00:34,320 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:41490, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:34,320 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 5/10 local region flush tasks. 2023-07-13 06:00:34,320 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] regionserver.HRegion(2446): Flush status journal for b06e3484b8c05bbdc046fc6b5386dddc: 2023-07-13 06:00:34,320 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 87846b634b85323bd205e28cff8a7ccc: 2023-07-13 06:00:34,321 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc. 2023-07-13 06:00:34,321 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04. 2023-07-13 06:00:34,321 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04. started... 2023-07-13 06:00:34,321 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc. 2023-07-13 06:00:34,321 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e16a1a1ea32dc6c137274851c98ece04: 2023-07-13 06:00:34,321 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04. 2023-07-13 06:00:34,321 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 6/10 local region flush tasks. 2023-07-13 06:00:34,320 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] regionserver.HRegion(2446): Flush status journal for ea36c09b963764399342ace600417093: 2023-07-13 06:00:34,321 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093. 2023-07-13 06:00:34,321 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 7/10 local region flush tasks. 2023-07-13 06:00:34,321 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 8/10 local region flush tasks. 2023-07-13 06:00:34,321 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 9/10 local region flush tasks. 2023-07-13 06:00:34,321 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 10/10 local region flush tasks. 2023-07-13 06:00:34,321 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 10 local region flush tasks. 2023-07-13 06:00:34,321 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:34,321 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testTablesWithSnapshots4' locally acquired 2023-07-13 06:00:34,321 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,35187,1689227983701' joining acquired barrier for procedure (testTablesWithSnapshots4) in zk 2023-07-13 06:00:34,412 DEBUG [master/jenkins-hbase20:0.Chore.1] quotas.QuotaObserverChore(336): testTablesWithSnapshots4 moved into observance of table space quota. 2023-07-13 06:00:34,423 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testTablesWithSnapshots4 2023-07-13 06:00:34,423 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testTablesWithSnapshots4/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:34,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testTablesWithSnapshots4/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:34,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:34,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:34,424 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testTablesWithSnapshots4 2023-07-13 06:00:34,424 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testTablesWithSnapshots4' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-13 06:00:34,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:34,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:34,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesWithSnapshots4 2023-07-13 06:00:34,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:34,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:34,425 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,35187,1689227983701' joining acquired barrier for procedure 'testTablesWithSnapshots4' on coordinator 2023-07-13 06:00:34,425 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5387bbfb[Count = 0] remaining members to acquire global barrier 2023-07-13 06:00:34,425 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testTablesWithSnapshots4' starting 'in-barrier' execution. 2023-07-13 06:00:34,425 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testTablesWithSnapshots4 2023-07-13 06:00:34,426 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testTablesWithSnapshots4 2023-07-13 06:00:34,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testTablesWithSnapshots4 2023-07-13 06:00:34,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testTablesWithSnapshots4 2023-07-13 06:00:34,426 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testTablesWithSnapshots4' received 'reached' from coordinator. 2023-07-13 06:00:34,426 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testTablesWithSnapshots4' locally completed 2023-07-13 06:00:34,426 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testTablesWithSnapshots4/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:34,426 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-13 06:00:34,426 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testTablesWithSnapshots4' completed for member 'jenkins-hbase20.apache.org,35187,1689227983701' in zk 2023-07-13 06:00:34,427 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testTablesWithSnapshots4' has notified controller of completion 2023-07-13 06:00:34,427 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-13 06:00:34,427 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testTablesWithSnapshots4' completed. 2023-07-13 06:00:34,428 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testTablesWithSnapshots4/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:34,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testTablesWithSnapshots4/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:34,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:34,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:34,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:34,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:34,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesWithSnapshots4 2023-07-13 06:00:34,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:34,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:34,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesWithSnapshots4 2023-07-13 06:00:34,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:34,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testTablesWithSnapshots4' member 'jenkins-hbase20.apache.org,35187,1689227983701': 2023-07-13 06:00:34,431 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,35187,1689227983701' released barrier for procedure'testTablesWithSnapshots4', counting down latch. Waiting for 0 more 2023-07-13 06:00:34,431 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testTablesWithSnapshots4' execution completed 2023-07-13 06:00:34,431 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-13 06:00:34,431 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-13 06:00:34,431 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testTablesWithSnapshots4 2023-07-13 06:00:34,431 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testTablesWithSnapshots4including nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-13 06:00:34,432 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testTablesWithSnapshots4 2023-07-13 06:00:34,432 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testTablesWithSnapshots4 2023-07-13 06:00:34,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testTablesWithSnapshots4 2023-07-13 06:00:34,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testTablesWithSnapshots4 2023-07-13 06:00:34,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:34,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:34,432 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-13 06:00:34,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testTablesWithSnapshots4 2023-07-13 06:00:34,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:34,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-13 06:00:34,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-13 06:00:34,433 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testTablesWithSnapshots4/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:34,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesWithSnapshots4 2023-07-13 06:00:34,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testTablesWithSnapshots4 2023-07-13 06:00:34,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:34,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesWithSnapshots4 2023-07-13 06:00:34,434 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testTablesWithSnapshots4/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:34,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:34,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:34,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesWithSnapshots4 2023-07-13 06:00:34,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:34,435 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testTablesWithSnapshots4/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:34,435 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testTablesWithSnapshots4 2023-07-13 06:00:34,436 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testTablesWithSnapshots4 2023-07-13 06:00:34,436 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testTablesWithSnapshots4/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:34,436 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testTablesWithSnapshots4 2023-07-13 06:00:34,436 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testTablesWithSnapshots4 2023-07-13 06:00:34,436 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testTablesWithSnapshots4 2023-07-13 06:00:34,436 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testTablesWithSnapshots4' 2023-07-13 06:00:34,436 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-13 06:00:34,436 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-13 06:00:34,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-13 06:00:34,436 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-13 06:00:34,436 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-13 06:00:34,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-13 06:00:34,437 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testTablesWithSnapshots4'' to complete. (max 20000 ms per retry) 2023-07-13 06:00:34,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-13 06:00:34,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-13 06:00:34,437 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-13 06:00:34,537 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-13 06:00:34,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-13 06:00:34,539 DEBUG [Listener at localhost.localdomain/45717] compactions.SortedCompactionPolicy(75): Selecting compaction from 2 store files, 0 compacting, 2 eligible, 16 blocking 2023-07-13 06:00:34,539 DEBUG [Listener at localhost.localdomain/45717] regionserver.HStore(1912): 2994958daca2f80926fd1799582c2b37/f1 is initiating major compaction (all files) 2023-07-13 06:00:34,539 INFO [Listener at localhost.localdomain/45717] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-13 06:00:34,539 INFO [Listener at localhost.localdomain/45717] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-13 06:00:34,540 INFO [Listener at localhost.localdomain/45717] regionserver.HRegion(2259): Starting compaction of 2994958daca2f80926fd1799582c2b37/f1 in testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37. 2023-07-13 06:00:34,540 INFO [Listener at localhost.localdomain/45717] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/2994958daca2f80926fd1799582c2b37/f1/3e409f745f004dcab1da0ad081befa99, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/2994958daca2f80926fd1799582c2b37/f1/e3550829b34146b591d67cb4c3543ecf] into tmpdir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/2994958daca2f80926fd1799582c2b37/.tmp, totalSize=238.8 K 2023-07-13 06:00:34,541 DEBUG [Listener at localhost.localdomain/45717] compactions.Compactor(207): Compacting 3e409f745f004dcab1da0ad081befa99, keycount=410, bloomtype=ROW, size=119.3 K, encoding=NONE, compression=NONE, seqNum=85, earliestPutTs=1689228026288 2023-07-13 06:00:34,542 DEBUG [Listener at localhost.localdomain/45717] compactions.Compactor(207): Compacting e3550829b34146b591d67cb4c3543ecf, keycount=410, bloomtype=ROW, size=119.5 K, encoding=NONE, compression=NONE, seqNum=170, earliestPutTs=1689228032214 2023-07-13 06:00:34,558 INFO [Listener at localhost.localdomain/45717] throttle.PressureAwareThroughputController(145): 2994958daca2f80926fd1799582c2b37#f1#compaction#24 average throughput is 36.98 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-13 06:00:34,974 DEBUG [Listener at localhost.localdomain/45717] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/2994958daca2f80926fd1799582c2b37/.tmp/f1/9a1f1f4c2bfc4dee88ed670a47334c8c as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/2994958daca2f80926fd1799582c2b37/f1/9a1f1f4c2bfc4dee88ed670a47334c8c 2023-07-13 06:00:34,979 INFO [Listener at localhost.localdomain/45717] regionserver.HStore(1652): Completed major compaction of 2 (all) file(s) in 2994958daca2f80926fd1799582c2b37/f1 of 2994958daca2f80926fd1799582c2b37 into 9a1f1f4c2bfc4dee88ed670a47334c8c(size=233.7 K), total size for store is 233.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-13 06:00:34,979 DEBUG [Listener at localhost.localdomain/45717] regionserver.HRegion(2289): Compaction status journal for 2994958daca2f80926fd1799582c2b37: 2023-07-13 06:00:34,979 DEBUG [Listener at localhost.localdomain/45717] compactions.SortedCompactionPolicy(75): Selecting compaction from 0 store files, 0 compacting, 0 eligible, 16 blocking 2023-07-13 06:00:34,979 DEBUG [Listener at localhost.localdomain/45717] compactions.SortedCompactionPolicy(75): Selecting compaction from 2 store files, 0 compacting, 2 eligible, 16 blocking 2023-07-13 06:00:34,980 DEBUG [Listener at localhost.localdomain/45717] regionserver.HStore(1912): 6817724a8c0ba54dffa0384c0e1b4834/f1 is initiating major compaction (all files) 2023-07-13 06:00:34,980 INFO [Listener at localhost.localdomain/45717] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-13 06:00:34,980 INFO [Listener at localhost.localdomain/45717] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-13 06:00:34,980 INFO [Listener at localhost.localdomain/45717] regionserver.HRegion(2259): Starting compaction of 6817724a8c0ba54dffa0384c0e1b4834/f1 in testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834. 2023-07-13 06:00:34,980 INFO [Listener at localhost.localdomain/45717] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/6817724a8c0ba54dffa0384c0e1b4834/f1/66ebca1e385b4bb6a4845c563267004a, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/6817724a8c0ba54dffa0384c0e1b4834/f1/37d03617d0a3478793a526fe0961f0e0] into tmpdir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/6817724a8c0ba54dffa0384c0e1b4834/.tmp, totalSize=238.3 K 2023-07-13 06:00:34,980 DEBUG [Listener at localhost.localdomain/45717] compactions.Compactor(207): Compacting 66ebca1e385b4bb6a4845c563267004a, keycount=409, bloomtype=ROW, size=119.0 K, encoding=NONE, compression=NONE, seqNum=85, earliestPutTs=1689228026290 2023-07-13 06:00:34,981 DEBUG [Listener at localhost.localdomain/45717] compactions.Compactor(207): Compacting 37d03617d0a3478793a526fe0961f0e0, keycount=409, bloomtype=ROW, size=119.2 K, encoding=NONE, compression=NONE, seqNum=170, earliestPutTs=1689228032218 2023-07-13 06:00:34,991 INFO [Listener at localhost.localdomain/45717] throttle.PressureAwareThroughputController(145): 6817724a8c0ba54dffa0384c0e1b4834#f1#compaction#25 average throughput is 73.78 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-13 06:00:35,265 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/2994958daca2f80926fd1799582c2b37/f1/3e409f745f004dcab1da0ad081befa99, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/2994958daca2f80926fd1799582c2b37/f1/e3550829b34146b591d67cb4c3543ecf] to archive 2023-07-13 06:00:35,266 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-07-13 06:00:35,268 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/2994958daca2f80926fd1799582c2b37/f1/3e409f745f004dcab1da0ad081befa99 to hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/archive/data/default/testTablesWithSnapshots4/2994958daca2f80926fd1799582c2b37/f1/3e409f745f004dcab1da0ad081befa99 2023-07-13 06:00:35,270 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/2994958daca2f80926fd1799582c2b37/f1/e3550829b34146b591d67cb4c3543ecf to hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/archive/data/default/testTablesWithSnapshots4/2994958daca2f80926fd1799582c2b37/f1/e3550829b34146b591d67cb4c3543ecf 2023-07-13 06:00:35,311 DEBUG [master/jenkins-hbase20:0.Chore.1] quotas.QuotaObserverChore(336): testTablesWithSnapshots4 moved into observance of table space quota. 2023-07-13 06:00:35,314 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x58f2bc25 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:35,317 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4051e6fd, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:35,321 DEBUG [hconnection-0x52bc557a-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:35,322 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:41506, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:35,400 DEBUG [Listener at localhost.localdomain/45717] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/6817724a8c0ba54dffa0384c0e1b4834/.tmp/f1/c9a27707e6aa42b68da2412246d0b5ed as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/6817724a8c0ba54dffa0384c0e1b4834/f1/c9a27707e6aa42b68da2412246d0b5ed 2023-07-13 06:00:35,407 INFO [Listener at localhost.localdomain/45717] regionserver.HStore(1652): Completed major compaction of 2 (all) file(s) in 6817724a8c0ba54dffa0384c0e1b4834/f1 of 6817724a8c0ba54dffa0384c0e1b4834 into c9a27707e6aa42b68da2412246d0b5ed(size=233.2 K), total size for store is 233.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-13 06:00:35,407 DEBUG [Listener at localhost.localdomain/45717] regionserver.HRegion(2289): Compaction status journal for 6817724a8c0ba54dffa0384c0e1b4834: 2023-07-13 06:00:35,408 DEBUG [Listener at localhost.localdomain/45717] compactions.SortedCompactionPolicy(75): Selecting compaction from 2 store files, 0 compacting, 2 eligible, 16 blocking 2023-07-13 06:00:35,408 DEBUG [Listener at localhost.localdomain/45717] regionserver.HStore(1912): 41c84bc63b9629db7bd4a197ebc93b69/f1 is initiating major compaction (all files) 2023-07-13 06:00:35,408 INFO [Listener at localhost.localdomain/45717] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-13 06:00:35,408 INFO [Listener at localhost.localdomain/45717] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-13 06:00:35,409 INFO [Listener at localhost.localdomain/45717] regionserver.HRegion(2259): Starting compaction of 41c84bc63b9629db7bd4a197ebc93b69/f1 in testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69. 2023-07-13 06:00:35,409 INFO [Listener at localhost.localdomain/45717] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/41c84bc63b9629db7bd4a197ebc93b69/f1/08b41d3cfcac4cf3af01961fbb311929, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/41c84bc63b9629db7bd4a197ebc93b69/f1/a3afa38e743544329100dabfb01fd8fd] into tmpdir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/41c84bc63b9629db7bd4a197ebc93b69/.tmp, totalSize=238.8 K 2023-07-13 06:00:35,410 DEBUG [Listener at localhost.localdomain/45717] compactions.Compactor(207): Compacting 08b41d3cfcac4cf3af01961fbb311929, keycount=410, bloomtype=ROW, size=119.3 K, encoding=NONE, compression=NONE, seqNum=85, earliestPutTs=1689228026287 2023-07-13 06:00:35,410 DEBUG [Listener at localhost.localdomain/45717] compactions.Compactor(207): Compacting a3afa38e743544329100dabfb01fd8fd, keycount=410, bloomtype=ROW, size=119.5 K, encoding=NONE, compression=NONE, seqNum=170, earliestPutTs=1689228032212 2023-07-13 06:00:35,421 INFO [Listener at localhost.localdomain/45717] throttle.PressureAwareThroughputController(145): 41c84bc63b9629db7bd4a197ebc93b69#f1#compaction#26 average throughput is 73.96 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-13 06:00:35,831 DEBUG [Listener at localhost.localdomain/45717] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/41c84bc63b9629db7bd4a197ebc93b69/.tmp/f1/cba06e75e7384c18bc3ffe12ba84f01d as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/41c84bc63b9629db7bd4a197ebc93b69/f1/cba06e75e7384c18bc3ffe12ba84f01d 2023-07-13 06:00:35,838 INFO [Listener at localhost.localdomain/45717] regionserver.HStore(1652): Completed major compaction of 2 (all) file(s) in 41c84bc63b9629db7bd4a197ebc93b69/f1 of 41c84bc63b9629db7bd4a197ebc93b69 into cba06e75e7384c18bc3ffe12ba84f01d(size=233.7 K), total size for store is 233.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-13 06:00:35,838 DEBUG [Listener at localhost.localdomain/45717] regionserver.HRegion(2289): Compaction status journal for 41c84bc63b9629db7bd4a197ebc93b69: 2023-07-13 06:00:35,838 DEBUG [Listener at localhost.localdomain/45717] compactions.SortedCompactionPolicy(75): Selecting compaction from 2 store files, 0 compacting, 2 eligible, 16 blocking 2023-07-13 06:00:35,839 DEBUG [Listener at localhost.localdomain/45717] regionserver.HStore(1912): 0d24aa4917843c73c4cba790560f2b8d/f1 is initiating major compaction (all files) 2023-07-13 06:00:35,839 INFO [Listener at localhost.localdomain/45717] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-13 06:00:35,839 INFO [Listener at localhost.localdomain/45717] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-13 06:00:35,839 INFO [Listener at localhost.localdomain/45717] regionserver.HRegion(2259): Starting compaction of 0d24aa4917843c73c4cba790560f2b8d/f1 in testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d. 2023-07-13 06:00:35,839 INFO [Listener at localhost.localdomain/45717] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/0d24aa4917843c73c4cba790560f2b8d/f1/af2f0b7204f343ee92c1eb2435b38351, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/0d24aa4917843c73c4cba790560f2b8d/f1/2ccdd294cede4d69b2ddedf7d61ce80e] into tmpdir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/0d24aa4917843c73c4cba790560f2b8d/.tmp, totalSize=238.8 K 2023-07-13 06:00:35,840 DEBUG [Listener at localhost.localdomain/45717] compactions.Compactor(207): Compacting af2f0b7204f343ee92c1eb2435b38351, keycount=410, bloomtype=ROW, size=119.3 K, encoding=NONE, compression=NONE, seqNum=85, earliestPutTs=1689228026288 2023-07-13 06:00:35,840 DEBUG [Listener at localhost.localdomain/45717] compactions.Compactor(207): Compacting 2ccdd294cede4d69b2ddedf7d61ce80e, keycount=410, bloomtype=ROW, size=119.5 K, encoding=NONE, compression=NONE, seqNum=170, earliestPutTs=1689228032215 2023-07-13 06:00:35,849 INFO [Listener at localhost.localdomain/45717] throttle.PressureAwareThroughputController(145): 0d24aa4917843c73c4cba790560f2b8d#f1#compaction#27 average throughput is 110.94 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-13 06:00:35,857 DEBUG [Listener at localhost.localdomain/45717] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/0d24aa4917843c73c4cba790560f2b8d/.tmp/f1/be6756d0921543ac8bac81a1f4345db2 as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/0d24aa4917843c73c4cba790560f2b8d/f1/be6756d0921543ac8bac81a1f4345db2 2023-07-13 06:00:35,862 INFO [Listener at localhost.localdomain/45717] regionserver.HStore(1652): Completed major compaction of 2 (all) file(s) in 0d24aa4917843c73c4cba790560f2b8d/f1 of 0d24aa4917843c73c4cba790560f2b8d into be6756d0921543ac8bac81a1f4345db2(size=233.7 K), total size for store is 233.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-13 06:00:35,862 DEBUG [Listener at localhost.localdomain/45717] regionserver.HRegion(2289): Compaction status journal for 0d24aa4917843c73c4cba790560f2b8d: 2023-07-13 06:00:35,862 DEBUG [Listener at localhost.localdomain/45717] compactions.SortedCompactionPolicy(75): Selecting compaction from 2 store files, 0 compacting, 2 eligible, 16 blocking 2023-07-13 06:00:35,862 DEBUG [Listener at localhost.localdomain/45717] regionserver.HStore(1912): 186edc75693ac04a787ff0d8059708f3/f1 is initiating major compaction (all files) 2023-07-13 06:00:35,863 INFO [Listener at localhost.localdomain/45717] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-13 06:00:35,863 INFO [Listener at localhost.localdomain/45717] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-13 06:00:35,863 INFO [Listener at localhost.localdomain/45717] regionserver.HRegion(2259): Starting compaction of 186edc75693ac04a787ff0d8059708f3/f1 in testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3. 2023-07-13 06:00:35,863 INFO [Listener at localhost.localdomain/45717] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/186edc75693ac04a787ff0d8059708f3/f1/78ea895f487c47b886a4dd0565aa64d5, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/186edc75693ac04a787ff0d8059708f3/f1/6d6410d899e64e10b85493e333f9aa1e] into tmpdir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/186edc75693ac04a787ff0d8059708f3/.tmp, totalSize=467.2 K 2023-07-13 06:00:35,863 DEBUG [Listener at localhost.localdomain/45717] compactions.Compactor(207): Compacting 78ea895f487c47b886a4dd0565aa64d5, keycount=818, bloomtype=ROW, size=233.4 K, encoding=NONE, compression=NONE, seqNum=85, earliestPutTs=1689228026290 2023-07-13 06:00:35,864 DEBUG [Listener at localhost.localdomain/45717] compactions.Compactor(207): Compacting 6d6410d899e64e10b85493e333f9aa1e, keycount=818, bloomtype=ROW, size=233.8 K, encoding=NONE, compression=NONE, seqNum=170, earliestPutTs=1689228032217 2023-07-13 06:00:35,881 INFO [Listener at localhost.localdomain/45717] throttle.PressureAwareThroughputController(145): 186edc75693ac04a787ff0d8059708f3#f1#compaction#28 average throughput is 49.19 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-13 06:00:36,249 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x16604c10 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:36,251 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@870f4b6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:36,253 DEBUG [hconnection-0xad3b3c9-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:36,255 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:41508, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:36,291 DEBUG [Listener at localhost.localdomain/45717] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/186edc75693ac04a787ff0d8059708f3/.tmp/f1/543b94d699e743adace436114a46e6e5 as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/186edc75693ac04a787ff0d8059708f3/f1/543b94d699e743adace436114a46e6e5 2023-07-13 06:00:36,299 INFO [Listener at localhost.localdomain/45717] regionserver.HStore(1652): Completed major compaction of 2 (all) file(s) in 186edc75693ac04a787ff0d8059708f3/f1 of 186edc75693ac04a787ff0d8059708f3 into 543b94d699e743adace436114a46e6e5(size=461.6 K), total size for store is 461.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-13 06:00:36,299 DEBUG [Listener at localhost.localdomain/45717] regionserver.HRegion(2289): Compaction status journal for 186edc75693ac04a787ff0d8059708f3: 2023-07-13 06:00:36,299 DEBUG [Listener at localhost.localdomain/45717] compactions.SortedCompactionPolicy(75): Selecting compaction from 2 store files, 0 compacting, 2 eligible, 16 blocking 2023-07-13 06:00:36,299 DEBUG [Listener at localhost.localdomain/45717] regionserver.HStore(1912): ea36c09b963764399342ace600417093/f1 is initiating major compaction (all files) 2023-07-13 06:00:36,300 INFO [Listener at localhost.localdomain/45717] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-13 06:00:36,300 INFO [Listener at localhost.localdomain/45717] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-13 06:00:36,301 INFO [Listener at localhost.localdomain/45717] regionserver.HRegion(2259): Starting compaction of ea36c09b963764399342ace600417093/f1 in testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093. 2023-07-13 06:00:36,301 INFO [Listener at localhost.localdomain/45717] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/ea36c09b963764399342ace600417093/f1/3a683e9afd184ad1ad1227e6d2337cd0, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/ea36c09b963764399342ace600417093/f1/5fea5c8d7df14a68aa60c36c5c08bd15] into tmpdir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/ea36c09b963764399342ace600417093/.tmp, totalSize=238.8 K 2023-07-13 06:00:36,301 DEBUG [Listener at localhost.localdomain/45717] compactions.Compactor(207): Compacting 3a683e9afd184ad1ad1227e6d2337cd0, keycount=410, bloomtype=ROW, size=119.3 K, encoding=NONE, compression=NONE, seqNum=85, earliestPutTs=1689228026287 2023-07-13 06:00:36,302 DEBUG [Listener at localhost.localdomain/45717] compactions.Compactor(207): Compacting 5fea5c8d7df14a68aa60c36c5c08bd15, keycount=410, bloomtype=ROW, size=119.5 K, encoding=NONE, compression=NONE, seqNum=170, earliestPutTs=1689228032214 2023-07-13 06:00:36,318 INFO [Listener at localhost.localdomain/45717] throttle.PressureAwareThroughputController(145): ea36c09b963764399342ace600417093#f1#compaction#29 average throughput is 24.65 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-13 06:00:36,328 DEBUG [Listener at localhost.localdomain/45717] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/ea36c09b963764399342ace600417093/.tmp/f1/fd57233595b341459de5da226082b567 as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/ea36c09b963764399342ace600417093/f1/fd57233595b341459de5da226082b567 2023-07-13 06:00:36,335 INFO [Listener at localhost.localdomain/45717] regionserver.HStore(1652): Completed major compaction of 2 (all) file(s) in ea36c09b963764399342ace600417093/f1 of ea36c09b963764399342ace600417093 into fd57233595b341459de5da226082b567(size=233.7 K), total size for store is 233.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-13 06:00:36,336 DEBUG [Listener at localhost.localdomain/45717] regionserver.HRegion(2289): Compaction status journal for ea36c09b963764399342ace600417093: 2023-07-13 06:00:36,336 DEBUG [Listener at localhost.localdomain/45717] compactions.SortedCompactionPolicy(75): Selecting compaction from 2 store files, 0 compacting, 2 eligible, 16 blocking 2023-07-13 06:00:36,336 DEBUG [Listener at localhost.localdomain/45717] regionserver.HStore(1912): 87846b634b85323bd205e28cff8a7ccc/f1 is initiating major compaction (all files) 2023-07-13 06:00:36,336 INFO [Listener at localhost.localdomain/45717] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-13 06:00:36,336 INFO [Listener at localhost.localdomain/45717] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-13 06:00:36,336 INFO [Listener at localhost.localdomain/45717] regionserver.HRegion(2259): Starting compaction of 87846b634b85323bd205e28cff8a7ccc/f1 in testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc. 2023-07-13 06:00:36,337 INFO [Listener at localhost.localdomain/45717] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/87846b634b85323bd205e28cff8a7ccc/f1/df7c07ab0bf2439cb03e7a21a814ddfc, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/87846b634b85323bd205e28cff8a7ccc/f1/2f6a2d63486d4b11ab778f53ea0f8c9a] into tmpdir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/87846b634b85323bd205e28cff8a7ccc/.tmp, totalSize=238.3 K 2023-07-13 06:00:36,337 DEBUG [Listener at localhost.localdomain/45717] compactions.Compactor(207): Compacting df7c07ab0bf2439cb03e7a21a814ddfc, keycount=409, bloomtype=ROW, size=119.0 K, encoding=NONE, compression=NONE, seqNum=85, earliestPutTs=1689228026289 2023-07-13 06:00:36,337 DEBUG [Listener at localhost.localdomain/45717] compactions.Compactor(207): Compacting 2f6a2d63486d4b11ab778f53ea0f8c9a, keycount=409, bloomtype=ROW, size=119.2 K, encoding=NONE, compression=NONE, seqNum=170, earliestPutTs=1689228032217 2023-07-13 06:00:36,353 INFO [Listener at localhost.localdomain/45717] throttle.PressureAwareThroughputController(145): 87846b634b85323bd205e28cff8a7ccc#f1#compaction#30 average throughput is 110.67 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-13 06:00:36,362 DEBUG [Listener at localhost.localdomain/45717] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/87846b634b85323bd205e28cff8a7ccc/.tmp/f1/d47b84f5ddc5494fba5987733b07ec70 as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/87846b634b85323bd205e28cff8a7ccc/f1/d47b84f5ddc5494fba5987733b07ec70 2023-07-13 06:00:36,368 DEBUG [master/jenkins-hbase20:0.Chore.1] quotas.QuotaObserverChore(336): testTablesWithSnapshots4 moved into observance of table space quota. 2023-07-13 06:00:36,368 INFO [Listener at localhost.localdomain/45717] regionserver.HStore(1652): Completed major compaction of 2 (all) file(s) in 87846b634b85323bd205e28cff8a7ccc/f1 of 87846b634b85323bd205e28cff8a7ccc into d47b84f5ddc5494fba5987733b07ec70(size=233.2 K), total size for store is 233.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-13 06:00:36,369 DEBUG [Listener at localhost.localdomain/45717] regionserver.HRegion(2289): Compaction status journal for 87846b634b85323bd205e28cff8a7ccc: 2023-07-13 06:00:36,369 DEBUG [Listener at localhost.localdomain/45717] compactions.SortedCompactionPolicy(75): Selecting compaction from 2 store files, 0 compacting, 2 eligible, 16 blocking 2023-07-13 06:00:36,369 DEBUG [Listener at localhost.localdomain/45717] regionserver.HStore(1912): b06e3484b8c05bbdc046fc6b5386dddc/f1 is initiating major compaction (all files) 2023-07-13 06:00:36,369 INFO [Listener at localhost.localdomain/45717] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-13 06:00:36,369 INFO [Listener at localhost.localdomain/45717] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-13 06:00:36,370 INFO [Listener at localhost.localdomain/45717] regionserver.HRegion(2259): Starting compaction of b06e3484b8c05bbdc046fc6b5386dddc/f1 in testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc. 2023-07-13 06:00:36,370 INFO [Listener at localhost.localdomain/45717] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/b06e3484b8c05bbdc046fc6b5386dddc/f1/77315930387547fcb82700980d49b594, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/b06e3484b8c05bbdc046fc6b5386dddc/f1/b4f4dc659c2e4211abd5d08f08eacbc9] into tmpdir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/b06e3484b8c05bbdc046fc6b5386dddc/.tmp, totalSize=238.8 K 2023-07-13 06:00:36,370 DEBUG [Listener at localhost.localdomain/45717] compactions.Compactor(207): Compacting 77315930387547fcb82700980d49b594, keycount=410, bloomtype=ROW, size=119.3 K, encoding=NONE, compression=NONE, seqNum=85, earliestPutTs=1689228026284 2023-07-13 06:00:36,371 DEBUG [Listener at localhost.localdomain/45717] compactions.Compactor(207): Compacting b4f4dc659c2e4211abd5d08f08eacbc9, keycount=410, bloomtype=ROW, size=119.5 K, encoding=NONE, compression=NONE, seqNum=170, earliestPutTs=1689228032211 2023-07-13 06:00:36,380 INFO [Listener at localhost.localdomain/45717] throttle.PressureAwareThroughputController(145): b06e3484b8c05bbdc046fc6b5386dddc#f1#compaction#31 average throughput is 110.94 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-13 06:00:36,797 DEBUG [Listener at localhost.localdomain/45717] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/b06e3484b8c05bbdc046fc6b5386dddc/.tmp/f1/3bab4db1b8bd4c298143b57eedf099f7 as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/b06e3484b8c05bbdc046fc6b5386dddc/f1/3bab4db1b8bd4c298143b57eedf099f7 2023-07-13 06:00:36,803 INFO [Listener at localhost.localdomain/45717] regionserver.HStore(1652): Completed major compaction of 2 (all) file(s) in b06e3484b8c05bbdc046fc6b5386dddc/f1 of b06e3484b8c05bbdc046fc6b5386dddc into 3bab4db1b8bd4c298143b57eedf099f7(size=233.7 K), total size for store is 233.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-13 06:00:36,803 DEBUG [Listener at localhost.localdomain/45717] regionserver.HRegion(2289): Compaction status journal for b06e3484b8c05bbdc046fc6b5386dddc: 2023-07-13 06:00:36,803 DEBUG [Listener at localhost.localdomain/45717] compactions.SortedCompactionPolicy(75): Selecting compaction from 2 store files, 0 compacting, 2 eligible, 16 blocking 2023-07-13 06:00:36,803 DEBUG [Listener at localhost.localdomain/45717] regionserver.HStore(1912): e16a1a1ea32dc6c137274851c98ece04/f1 is initiating major compaction (all files) 2023-07-13 06:00:36,803 INFO [Listener at localhost.localdomain/45717] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-13 06:00:36,803 INFO [Listener at localhost.localdomain/45717] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-13 06:00:36,803 INFO [Listener at localhost.localdomain/45717] regionserver.HRegion(2259): Starting compaction of e16a1a1ea32dc6c137274851c98ece04/f1 in testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04. 2023-07-13 06:00:36,804 INFO [Listener at localhost.localdomain/45717] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/e16a1a1ea32dc6c137274851c98ece04/f1/8d1735075c04430f88c4ee7785534ff9, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/e16a1a1ea32dc6c137274851c98ece04/f1/0c78701355e6406ca279301bab655166] into tmpdir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/e16a1a1ea32dc6c137274851c98ece04/.tmp, totalSize=238.8 K 2023-07-13 06:00:36,804 DEBUG [Listener at localhost.localdomain/45717] compactions.Compactor(207): Compacting 8d1735075c04430f88c4ee7785534ff9, keycount=410, bloomtype=ROW, size=119.3 K, encoding=NONE, compression=NONE, seqNum=85, earliestPutTs=1689228026289 2023-07-13 06:00:36,805 DEBUG [Listener at localhost.localdomain/45717] compactions.Compactor(207): Compacting 0c78701355e6406ca279301bab655166, keycount=410, bloomtype=ROW, size=119.5 K, encoding=NONE, compression=NONE, seqNum=170, earliestPutTs=1689228032216 2023-07-13 06:00:36,814 INFO [Listener at localhost.localdomain/45717] throttle.PressureAwareThroughputController(145): e16a1a1ea32dc6c137274851c98ece04#f1#compaction#32 average throughput is 110.94 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-13 06:00:37,233 DEBUG [Listener at localhost.localdomain/45717] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/e16a1a1ea32dc6c137274851c98ece04/.tmp/f1/2394a9437b744c72b8cf8f0a9275f3a0 as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/e16a1a1ea32dc6c137274851c98ece04/f1/2394a9437b744c72b8cf8f0a9275f3a0 2023-07-13 06:00:37,241 INFO [Listener at localhost.localdomain/45717] regionserver.HStore(1652): Completed major compaction of 2 (all) file(s) in e16a1a1ea32dc6c137274851c98ece04/f1 of e16a1a1ea32dc6c137274851c98ece04 into 2394a9437b744c72b8cf8f0a9275f3a0(size=233.7 K), total size for store is 233.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-13 06:00:37,241 DEBUG [Listener at localhost.localdomain/45717] regionserver.HRegion(2289): Compaction status journal for e16a1a1ea32dc6c137274851c98ece04: 2023-07-13 06:00:37,241 INFO [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots(326): Checking for quota violation 2023-07-13 06:00:37,242 INFO [Listener at localhost.localdomain/45717] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-13 06:00:37,244 INFO [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$10(342): 2535065/3145728 SpaceQuotaStatus[policy=Optional.empty, inViolation=false] 2023-07-13 06:00:37,304 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x541becf1 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:37,306 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@45168407, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:37,308 DEBUG [hconnection-0x5ba5328d-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:37,310 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:41514, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:37,396 DEBUG [master/jenkins-hbase20:0.Chore.1] quotas.QuotaObserverChore(336): testTablesWithSnapshots4 moved into observance of table space quota. 2023-07-13 06:00:38,241 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x07578a92 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:38,244 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@78ff65fc, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:38,245 INFO [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$10(342): 2508523/3145728 SpaceQuotaStatus[policy=Optional.empty, inViolation=false] 2023-07-13 06:00:38,245 DEBUG [hconnection-0x721a4548-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:38,246 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:41520, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:39,248 INFO [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$10(342): 2508523/3145728 SpaceQuotaStatus[policy=Optional.empty, inViolation=false] 2023-07-13 06:00:39,311 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x0fbfe2ca to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:39,314 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3d1374a1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:39,315 DEBUG [hconnection-0x622b4bab-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:39,317 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:46678, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:40,225 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x60fc217a to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:40,229 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4dbbf990, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:40,231 DEBUG [hconnection-0x118b4ffd-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:40,233 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:46692, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:40,250 INFO [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$10(342): 2508523/3145728 SpaceQuotaStatus[policy=Optional.empty, inViolation=false] 2023-07-13 06:00:40,322 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/6817724a8c0ba54dffa0384c0e1b4834/f1/66ebca1e385b4bb6a4845c563267004a, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/6817724a8c0ba54dffa0384c0e1b4834/f1/37d03617d0a3478793a526fe0961f0e0] to archive 2023-07-13 06:00:40,323 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-07-13 06:00:40,326 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/6817724a8c0ba54dffa0384c0e1b4834/f1/66ebca1e385b4bb6a4845c563267004a to hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/archive/data/default/testTablesWithSnapshots4/6817724a8c0ba54dffa0384c0e1b4834/f1/66ebca1e385b4bb6a4845c563267004a 2023-07-13 06:00:40,327 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/6817724a8c0ba54dffa0384c0e1b4834/f1/37d03617d0a3478793a526fe0961f0e0 to hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/archive/data/default/testTablesWithSnapshots4/6817724a8c0ba54dffa0384c0e1b4834/f1/37d03617d0a3478793a526fe0961f0e0 2023-07-13 06:00:40,337 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/41c84bc63b9629db7bd4a197ebc93b69/f1/08b41d3cfcac4cf3af01961fbb311929, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/41c84bc63b9629db7bd4a197ebc93b69/f1/a3afa38e743544329100dabfb01fd8fd] to archive 2023-07-13 06:00:40,338 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-07-13 06:00:40,341 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/41c84bc63b9629db7bd4a197ebc93b69/f1/08b41d3cfcac4cf3af01961fbb311929 to hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/archive/data/default/testTablesWithSnapshots4/41c84bc63b9629db7bd4a197ebc93b69/f1/08b41d3cfcac4cf3af01961fbb311929 2023-07-13 06:00:40,342 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/41c84bc63b9629db7bd4a197ebc93b69/f1/a3afa38e743544329100dabfb01fd8fd to hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/archive/data/default/testTablesWithSnapshots4/41c84bc63b9629db7bd4a197ebc93b69/f1/a3afa38e743544329100dabfb01fd8fd 2023-07-13 06:00:40,351 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/0d24aa4917843c73c4cba790560f2b8d/f1/af2f0b7204f343ee92c1eb2435b38351, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/0d24aa4917843c73c4cba790560f2b8d/f1/2ccdd294cede4d69b2ddedf7d61ce80e] to archive 2023-07-13 06:00:40,352 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-07-13 06:00:40,354 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/0d24aa4917843c73c4cba790560f2b8d/f1/af2f0b7204f343ee92c1eb2435b38351 to hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/archive/data/default/testTablesWithSnapshots4/0d24aa4917843c73c4cba790560f2b8d/f1/af2f0b7204f343ee92c1eb2435b38351 2023-07-13 06:00:40,355 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/0d24aa4917843c73c4cba790560f2b8d/f1/2ccdd294cede4d69b2ddedf7d61ce80e to hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/archive/data/default/testTablesWithSnapshots4/0d24aa4917843c73c4cba790560f2b8d/f1/2ccdd294cede4d69b2ddedf7d61ce80e 2023-07-13 06:00:40,365 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/186edc75693ac04a787ff0d8059708f3/f1/78ea895f487c47b886a4dd0565aa64d5, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/186edc75693ac04a787ff0d8059708f3/f1/6d6410d899e64e10b85493e333f9aa1e] to archive 2023-07-13 06:00:40,366 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-07-13 06:00:40,368 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/186edc75693ac04a787ff0d8059708f3/f1/78ea895f487c47b886a4dd0565aa64d5 to hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/archive/data/default/testTablesWithSnapshots4/186edc75693ac04a787ff0d8059708f3/f1/78ea895f487c47b886a4dd0565aa64d5 2023-07-13 06:00:40,369 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/186edc75693ac04a787ff0d8059708f3/f1/6d6410d899e64e10b85493e333f9aa1e to hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/archive/data/default/testTablesWithSnapshots4/186edc75693ac04a787ff0d8059708f3/f1/6d6410d899e64e10b85493e333f9aa1e 2023-07-13 06:00:40,379 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/ea36c09b963764399342ace600417093/f1/3a683e9afd184ad1ad1227e6d2337cd0, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/ea36c09b963764399342ace600417093/f1/5fea5c8d7df14a68aa60c36c5c08bd15] to archive 2023-07-13 06:00:40,380 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-07-13 06:00:40,382 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/ea36c09b963764399342ace600417093/f1/3a683e9afd184ad1ad1227e6d2337cd0 to hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/archive/data/default/testTablesWithSnapshots4/ea36c09b963764399342ace600417093/f1/3a683e9afd184ad1ad1227e6d2337cd0 2023-07-13 06:00:40,383 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/ea36c09b963764399342ace600417093/f1/5fea5c8d7df14a68aa60c36c5c08bd15 to hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/archive/data/default/testTablesWithSnapshots4/ea36c09b963764399342ace600417093/f1/5fea5c8d7df14a68aa60c36c5c08bd15 2023-07-13 06:00:40,394 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/87846b634b85323bd205e28cff8a7ccc/f1/df7c07ab0bf2439cb03e7a21a814ddfc, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/87846b634b85323bd205e28cff8a7ccc/f1/2f6a2d63486d4b11ab778f53ea0f8c9a] to archive 2023-07-13 06:00:40,395 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-07-13 06:00:40,397 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/87846b634b85323bd205e28cff8a7ccc/f1/df7c07ab0bf2439cb03e7a21a814ddfc to hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/archive/data/default/testTablesWithSnapshots4/87846b634b85323bd205e28cff8a7ccc/f1/df7c07ab0bf2439cb03e7a21a814ddfc 2023-07-13 06:00:40,398 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/87846b634b85323bd205e28cff8a7ccc/f1/2f6a2d63486d4b11ab778f53ea0f8c9a to hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/archive/data/default/testTablesWithSnapshots4/87846b634b85323bd205e28cff8a7ccc/f1/2f6a2d63486d4b11ab778f53ea0f8c9a 2023-07-13 06:00:40,409 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/b06e3484b8c05bbdc046fc6b5386dddc/f1/77315930387547fcb82700980d49b594, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/b06e3484b8c05bbdc046fc6b5386dddc/f1/b4f4dc659c2e4211abd5d08f08eacbc9] to archive 2023-07-13 06:00:40,409 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-07-13 06:00:40,411 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/b06e3484b8c05bbdc046fc6b5386dddc/f1/77315930387547fcb82700980d49b594 to hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/archive/data/default/testTablesWithSnapshots4/b06e3484b8c05bbdc046fc6b5386dddc/f1/77315930387547fcb82700980d49b594 2023-07-13 06:00:40,413 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/b06e3484b8c05bbdc046fc6b5386dddc/f1/b4f4dc659c2e4211abd5d08f08eacbc9 to hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/archive/data/default/testTablesWithSnapshots4/b06e3484b8c05bbdc046fc6b5386dddc/f1/b4f4dc659c2e4211abd5d08f08eacbc9 2023-07-13 06:00:40,422 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/e16a1a1ea32dc6c137274851c98ece04/f1/8d1735075c04430f88c4ee7785534ff9, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/e16a1a1ea32dc6c137274851c98ece04/f1/0c78701355e6406ca279301bab655166] to archive 2023-07-13 06:00:40,423 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-07-13 06:00:40,425 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/e16a1a1ea32dc6c137274851c98ece04/f1/8d1735075c04430f88c4ee7785534ff9 to hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/archive/data/default/testTablesWithSnapshots4/e16a1a1ea32dc6c137274851c98ece04/f1/8d1735075c04430f88c4ee7785534ff9 2023-07-13 06:00:40,427 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/e16a1a1ea32dc6c137274851c98ece04/f1/0c78701355e6406ca279301bab655166 to hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/archive/data/default/testTablesWithSnapshots4/e16a1a1ea32dc6c137274851c98ece04/f1/0c78701355e6406ca279301bab655166 2023-07-13 06:00:41,228 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x790afb59 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:41,231 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@34d4a1fb, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:41,234 DEBUG [hconnection-0x1f519239-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:41,235 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:46698, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:41,251 INFO [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$10(342): 2508523/3145728 SpaceQuotaStatus[policy=Optional.empty, inViolation=false] 2023-07-13 06:00:41,334 DEBUG [master/jenkins-hbase20:0.Chore.1] quotas.QuotaObserverChore(348): testTablesWithSnapshots4 moved into violation of table space quota with policy of NO_INSERTS 2023-07-13 06:00:42,245 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x68c2fdd6 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:42,247 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6e74de28, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:42,249 DEBUG [hconnection-0x4ca86bd4-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:42,250 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:46700, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:42,252 INFO [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$10(342): 3602162/3145728 SpaceQuotaStatus[policy=Optional[NO_INSERTS], inViolation=true] 2023-07-13 06:00:42,266 INFO [Listener at localhost.localdomain/45717] hbase.ResourceChecker(175): after: quotas.TestSpaceQuotasWithSnapshots#testTablesWithSnapshots Thread=263 (was 259) - Thread LEAK? -, OpenFileDescriptor=611 (was 557) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=128 (was 158), ProcessCount=168 (was 166) - ProcessCount LEAK? -, AvailableMemoryMB=6529 (was 6745) 2023-07-13 06:00:42,278 INFO [Listener at localhost.localdomain/45717] hbase.ResourceChecker(147): before: quotas.TestSpaceQuotasWithSnapshots#testTablesInheritSnapshotSize Thread=263, OpenFileDescriptor=611, MaxFileDescriptor=60000, SystemLoadAverage=128, ProcessCount=168, AvailableMemoryMB=6528 2023-07-13 06:00:42,281 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.HMaster$4(2112): Client=jenkins//148.251.75.209 create 'testTablesInheritSnapshotSize5', {NAME => 'f1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-13 06:00:42,281 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testTablesInheritSnapshotSize5 2023-07-13 06:00:42,283 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=49, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testTablesInheritSnapshotSize5 execute state=CREATE_TABLE_PRE_OPERATION 2023-07-13 06:00:42,283 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(700): Client=jenkins//148.251.75.209 procedure request for creating table: namespace: "default" qualifier: "testTablesInheritSnapshotSize5" procId is: 49 2023-07-13 06:00:42,283 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=49, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testTablesInheritSnapshotSize5 execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-13 06:00:42,284 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1230): Checking to see if procedure is done pid=49 2023-07-13 06:00:42,285 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testTablesInheritSnapshotSize5/d8ad6096a0579ab248b74b5f53c0c5f3 2023-07-13 06:00:42,285 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testTablesInheritSnapshotSize5/d8ad6096a0579ab248b74b5f53c0c5f3 empty. 2023-07-13 06:00:42,285 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testTablesInheritSnapshotSize5/d8ad6096a0579ab248b74b5f53c0c5f3 2023-07-13 06:00:42,285 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testTablesInheritSnapshotSize5 regions 2023-07-13 06:00:42,292 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp/data/default/testTablesInheritSnapshotSize5/.tabledesc/.tableinfo.0000000001 2023-07-13 06:00:42,293 INFO [RegionOpenAndInit-testTablesInheritSnapshotSize5-pool-0] regionserver.HRegion(7675): creating {ENCODED => d8ad6096a0579ab248b74b5f53c0c5f3, NAME => 'testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testTablesInheritSnapshotSize5', {NAME => 'f1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.tmp 2023-07-13 06:00:42,385 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1230): Checking to see if procedure is done pid=49 2023-07-13 06:00:42,586 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1230): Checking to see if procedure is done pid=49 2023-07-13 06:00:42,700 DEBUG [RegionOpenAndInit-testTablesInheritSnapshotSize5-pool-0] regionserver.HRegion(866): Instantiated testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 06:00:42,700 DEBUG [RegionOpenAndInit-testTablesInheritSnapshotSize5-pool-0] regionserver.HRegion(1604): Closing d8ad6096a0579ab248b74b5f53c0c5f3, disabling compactions & flushes 2023-07-13 06:00:42,700 INFO [RegionOpenAndInit-testTablesInheritSnapshotSize5-pool-0] regionserver.HRegion(1626): Closing region testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3. 2023-07-13 06:00:42,700 DEBUG [RegionOpenAndInit-testTablesInheritSnapshotSize5-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3. 2023-07-13 06:00:42,700 DEBUG [RegionOpenAndInit-testTablesInheritSnapshotSize5-pool-0] regionserver.HRegion(1714): Acquired close lock on testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3. after waiting 0 ms 2023-07-13 06:00:42,700 DEBUG [RegionOpenAndInit-testTablesInheritSnapshotSize5-pool-0] regionserver.HRegion(1724): Updates disabled for region testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3. 2023-07-13 06:00:42,700 INFO [RegionOpenAndInit-testTablesInheritSnapshotSize5-pool-0] regionserver.HRegion(1838): Closed testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3. 2023-07-13 06:00:42,701 DEBUG [RegionOpenAndInit-testTablesInheritSnapshotSize5-pool-0] regionserver.HRegion(1558): Region close journal for d8ad6096a0579ab248b74b5f53c0c5f3: 2023-07-13 06:00:42,705 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=49, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testTablesInheritSnapshotSize5 execute state=CREATE_TABLE_ADD_TO_META 2023-07-13 06:00:42,707 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689228042707"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1689228042707"}]},"ts":"1689228042707"} 2023-07-13 06:00:42,709 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-13 06:00:42,710 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=49, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testTablesInheritSnapshotSize5 execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-13 06:00:42,710 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testTablesInheritSnapshotSize5","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689228042710"}]},"ts":"1689228042710"} 2023-07-13 06:00:42,711 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testTablesInheritSnapshotSize5, state=ENABLING in hbase:meta 2023-07-13 06:00:42,714 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testTablesInheritSnapshotSize5, region=d8ad6096a0579ab248b74b5f53c0c5f3, ASSIGN}] 2023-07-13 06:00:42,716 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testTablesInheritSnapshotSize5, region=d8ad6096a0579ab248b74b5f53c0c5f3, ASSIGN 2023-07-13 06:00:42,717 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testTablesInheritSnapshotSize5, region=d8ad6096a0579ab248b74b5f53c0c5f3, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,35187,1689227983701; forceNewPlan=false, retain=false 2023-07-13 06:00:42,869 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=d8ad6096a0579ab248b74b5f53c0c5f3, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:42,870 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689228042869"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1689228042869"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1689228042869"}]},"ts":"1689228042869"} 2023-07-13 06:00:42,874 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; OpenRegionProcedure d8ad6096a0579ab248b74b5f53c0c5f3, server=jenkins-hbase20.apache.org,35187,1689227983701}] 2023-07-13 06:00:42,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1230): Checking to see if procedure is done pid=49 2023-07-13 06:00:43,032 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(130): Open testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3. 2023-07-13 06:00:43,032 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d8ad6096a0579ab248b74b5f53c0c5f3, NAME => 'testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3.', STARTKEY => '', ENDKEY => ''} 2023-07-13 06:00:43,033 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testTablesInheritSnapshotSize5 d8ad6096a0579ab248b74b5f53c0c5f3 2023-07-13 06:00:43,033 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-13 06:00:43,033 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for d8ad6096a0579ab248b74b5f53c0c5f3 2023-07-13 06:00:43,033 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for d8ad6096a0579ab248b74b5f53c0c5f3 2023-07-13 06:00:43,034 INFO [StoreOpener-d8ad6096a0579ab248b74b5f53c0c5f3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region d8ad6096a0579ab248b74b5f53c0c5f3 2023-07-13 06:00:43,036 DEBUG [StoreOpener-d8ad6096a0579ab248b74b5f53c0c5f3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesInheritSnapshotSize5/d8ad6096a0579ab248b74b5f53c0c5f3/f1 2023-07-13 06:00:43,037 DEBUG [StoreOpener-d8ad6096a0579ab248b74b5f53c0c5f3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesInheritSnapshotSize5/d8ad6096a0579ab248b74b5f53c0c5f3/f1 2023-07-13 06:00:43,037 INFO [StoreOpener-d8ad6096a0579ab248b74b5f53c0c5f3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d8ad6096a0579ab248b74b5f53c0c5f3 columnFamilyName f1 2023-07-13 06:00:43,038 INFO [StoreOpener-d8ad6096a0579ab248b74b5f53c0c5f3-1] regionserver.HStore(310): Store=d8ad6096a0579ab248b74b5f53c0c5f3/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-13 06:00:43,039 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesInheritSnapshotSize5/d8ad6096a0579ab248b74b5f53c0c5f3 2023-07-13 06:00:43,039 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesInheritSnapshotSize5/d8ad6096a0579ab248b74b5f53c0c5f3 2023-07-13 06:00:43,043 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for d8ad6096a0579ab248b74b5f53c0c5f3 2023-07-13 06:00:43,046 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesInheritSnapshotSize5/d8ad6096a0579ab248b74b5f53c0c5f3/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-13 06:00:43,047 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened d8ad6096a0579ab248b74b5f53c0c5f3; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75047696, jitterRate=0.11829781532287598}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-13 06:00:43,047 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for d8ad6096a0579ab248b74b5f53c0c5f3: 2023-07-13 06:00:43,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3., pid=51, masterSystemTime=1689228043029 2023-07-13 06:00:43,049 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3. 2023-07-13 06:00:43,049 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(158): Opened testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3. 2023-07-13 06:00:43,050 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=d8ad6096a0579ab248b74b5f53c0c5f3, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:43,050 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1689228043050"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1689228043050"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1689228043050"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1689228043050"}]},"ts":"1689228043050"} 2023-07-13 06:00:43,054 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-07-13 06:00:43,054 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; OpenRegionProcedure d8ad6096a0579ab248b74b5f53c0c5f3, server=jenkins-hbase20.apache.org,35187,1689227983701 in 178 msec 2023-07-13 06:00:43,057 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-07-13 06:00:43,057 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testTablesInheritSnapshotSize5, region=d8ad6096a0579ab248b74b5f53c0c5f3, ASSIGN in 340 msec 2023-07-13 06:00:43,057 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=49, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testTablesInheritSnapshotSize5 execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-13 06:00:43,058 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testTablesInheritSnapshotSize5","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1689228043058"}]},"ts":"1689228043058"} 2023-07-13 06:00:43,059 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testTablesInheritSnapshotSize5, state=ENABLED in hbase:meta 2023-07-13 06:00:43,062 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=49, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testTablesInheritSnapshotSize5 execute state=CREATE_TABLE_POST_OPERATION 2023-07-13 06:00:43,064 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; CreateTableProcedure table=testTablesInheritSnapshotSize5 in 781 msec 2023-07-13 06:00:43,242 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x1a274eac to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:43,246 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@77433f27, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:43,249 DEBUG [hconnection-0x5a843093-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:43,253 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:46702, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:43,388 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1230): Checking to see if procedure is done pid=49 2023-07-13 06:00:43,389 INFO [Listener at localhost.localdomain/45717] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testTablesInheritSnapshotSize5, procId: 49 completed 2023-07-13 06:00:43,389 INFO [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots(112): Writing data 2023-07-13 06:00:43,568 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests(539): Data was written to HBase 2023-07-13 06:00:43,570 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-13 06:00:43,571 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] procedure.ProcedureCoordinator(165): Submitting procedure testTablesInheritSnapshotSize5 2023-07-13 06:00:43,571 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testTablesInheritSnapshotSize5' 2023-07-13 06:00:43,571 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-13 06:00:43,572 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testTablesInheritSnapshotSize5' starting 'acquire' 2023-07-13 06:00:43,572 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testTablesInheritSnapshotSize5', kicking off acquire phase on members. 2023-07-13 06:00:43,572 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testTablesInheritSnapshotSize5 2023-07-13 06:00:43,572 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testTablesInheritSnapshotSize5 2023-07-13 06:00:43,573 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testTablesInheritSnapshotSize5/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:43,573 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-13 06:00:43,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-13 06:00:43,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-13 06:00:43,573 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testTablesInheritSnapshotSize5/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:43,573 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-13 06:00:43,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testTablesInheritSnapshotSize5 2023-07-13 06:00:43,574 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testTablesInheritSnapshotSize5 2023-07-13 06:00:43,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-13 06:00:43,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testTablesInheritSnapshotSize5 2023-07-13 06:00:43,574 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testTablesInheritSnapshotSize5 2023-07-13 06:00:43,574 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testTablesInheritSnapshotSize5 2023-07-13 06:00:43,574 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testTablesInheritSnapshotSize5' with timeout 60000ms 2023-07-13 06:00:43,574 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-13 06:00:43,575 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testTablesInheritSnapshotSize5' starting 'acquire' stage 2023-07-13 06:00:43,575 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-13 06:00:43,575 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-13 06:00:43,575 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3. 2023-07-13 06:00:43,575 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3. started... 2023-07-13 06:00:43,575 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d8ad6096a0579ab248b74b5f53c0c5f3 1/1 column families, dataSize=2.22 MB heapSize=2.81 MB 2023-07-13 06:00:43,997 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.22 MB at sequenceid=165 (bloomFilter=true), to=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesInheritSnapshotSize5/d8ad6096a0579ab248b74b5f53c0c5f3/.tmp/f1/209055e1a9db425fa87efcf218ca5d9d 2023-07-13 06:00:44,005 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesInheritSnapshotSize5/d8ad6096a0579ab248b74b5f53c0c5f3/.tmp/f1/209055e1a9db425fa87efcf218ca5d9d as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesInheritSnapshotSize5/d8ad6096a0579ab248b74b5f53c0c5f3/f1/209055e1a9db425fa87efcf218ca5d9d 2023-07-13 06:00:44,010 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesInheritSnapshotSize5/d8ad6096a0579ab248b74b5f53c0c5f3/f1/209055e1a9db425fa87efcf218ca5d9d, entries=8192, sequenceid=165, filesize=2.3 M 2023-07-13 06:00:44,010 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.22 MB/2325418, heapSize ~2.81 MB/2949360, currentSize=0 B/0 for d8ad6096a0579ab248b74b5f53c0c5f3 in 435ms, sequenceid=165, compaction requested=false 2023-07-13 06:00:44,010 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testTablesInheritSnapshotSize5' 2023-07-13 06:00:44,011 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d8ad6096a0579ab248b74b5f53c0c5f3: 2023-07-13 06:00:44,011 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3. 2023-07-13 06:00:44,011 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-13 06:00:44,011 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-13 06:00:44,011 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:44,011 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testTablesInheritSnapshotSize5' locally acquired 2023-07-13 06:00:44,011 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,35187,1689227983701' joining acquired barrier for procedure (testTablesInheritSnapshotSize5) in zk 2023-07-13 06:00:44,012 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5 2023-07-13 06:00:44,012 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testTablesInheritSnapshotSize5/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:44,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testTablesInheritSnapshotSize5/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:44,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:44,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:44,013 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5 2023-07-13 06:00:44,013 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testTablesInheritSnapshotSize5' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-13 06:00:44,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:44,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:44,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesInheritSnapshotSize5 2023-07-13 06:00:44,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:44,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:44,014 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,35187,1689227983701' joining acquired barrier for procedure 'testTablesInheritSnapshotSize5' on coordinator 2023-07-13 06:00:44,014 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@76d279a2[Count = 0] remaining members to acquire global barrier 2023-07-13 06:00:44,014 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testTablesInheritSnapshotSize5' starting 'in-barrier' execution. 2023-07-13 06:00:44,014 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5 2023-07-13 06:00:44,015 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5 2023-07-13 06:00:44,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5 2023-07-13 06:00:44,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5 2023-07-13 06:00:44,015 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testTablesInheritSnapshotSize5' received 'reached' from coordinator. 2023-07-13 06:00:44,015 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testTablesInheritSnapshotSize5' locally completed 2023-07-13 06:00:44,015 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testTablesInheritSnapshotSize5' completed for member 'jenkins-hbase20.apache.org,35187,1689227983701' in zk 2023-07-13 06:00:44,015 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:44,015 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-13 06:00:44,017 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testTablesInheritSnapshotSize5' has notified controller of completion 2023-07-13 06:00:44,017 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:44,017 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-13 06:00:44,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:44,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:44,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:44,017 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testTablesInheritSnapshotSize5' completed. 2023-07-13 06:00:44,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:44,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:44,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesInheritSnapshotSize5 2023-07-13 06:00:44,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:44,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:44,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesInheritSnapshotSize5 2023-07-13 06:00:44,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:44,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testTablesInheritSnapshotSize5' member 'jenkins-hbase20.apache.org,35187,1689227983701': 2023-07-13 06:00:44,020 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,35187,1689227983701' released barrier for procedure'testTablesInheritSnapshotSize5', counting down latch. Waiting for 0 more 2023-07-13 06:00:44,020 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testTablesInheritSnapshotSize5' execution completed 2023-07-13 06:00:44,020 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-13 06:00:44,020 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-13 06:00:44,020 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testTablesInheritSnapshotSize5 2023-07-13 06:00:44,020 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testTablesInheritSnapshotSize5including nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-13 06:00:44,021 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testTablesInheritSnapshotSize5 2023-07-13 06:00:44,021 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testTablesInheritSnapshotSize5 2023-07-13 06:00:44,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testTablesInheritSnapshotSize5 2023-07-13 06:00:44,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testTablesInheritSnapshotSize5 2023-07-13 06:00:44,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testTablesInheritSnapshotSize5 2023-07-13 06:00:44,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:44,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:44,021 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-13 06:00:44,021 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-13 06:00:44,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-13 06:00:44,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:44,022 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testTablesInheritSnapshotSize5/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:44,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testTablesInheritSnapshotSize5 2023-07-13 06:00:44,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesInheritSnapshotSize5 2023-07-13 06:00:44,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:44,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesInheritSnapshotSize5 2023-07-13 06:00:44,023 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:44,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:44,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:44,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesInheritSnapshotSize5 2023-07-13 06:00:44,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:44,025 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testTablesInheritSnapshotSize5/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:44,025 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testTablesInheritSnapshotSize5 2023-07-13 06:00:44,025 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-13 06:00:44,025 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testTablesInheritSnapshotSize5 2023-07-13 06:00:44,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-13 06:00:44,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-13 06:00:44,025 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-13 06:00:44,026 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testTablesInheritSnapshotSize5 because node does not exist (not an error) 2023-07-13 06:00:44,025 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testTablesInheritSnapshotSize5' 2023-07-13 06:00:44,026 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-13 06:00:44,025 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-13 06:00:44,025 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:44,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-13 06:00:44,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-13 06:00:44,027 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testTablesInheritSnapshotSize5'' to complete. (max 20000 ms per retry) 2023-07-13 06:00:44,027 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-13 06:00:44,027 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5 2023-07-13 06:00:44,027 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5 2023-07-13 06:00:44,027 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testTablesInheritSnapshotSize5 2023-07-13 06:00:44,127 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-13 06:00:44,128 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-13 06:00:44,128 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests(542): Data flushed to disk 2023-07-13 06:00:44,128 INFO [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots(121): Waiting until table size reflects written data 2023-07-13 06:00:44,129 INFO [Listener at localhost.localdomain/45717] hbase.Waiter(180): Waiting up to [30,000] milli-secs(wait.for.ratio=[1]) 2023-07-13 06:00:44,130 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for testTablesInheritSnapshotSize5: null 2023-07-13 06:00:44,228 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x2659fe5b to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:44,235 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@10b6dd5a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:44,237 DEBUG [hconnection-0x19c8b2a9-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:44,239 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:46706, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:44,316 DEBUG [master/jenkins-hbase20:0.Chore.1] quotas.QuotaObserverChore(336): testTablesInheritSnapshotSize5 moved into observance of table space quota. 2023-07-13 06:00:44,632 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for testTablesInheritSnapshotSize5: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=2.25 MB/1 GB] 2023-07-13 06:00:44,632 INFO [Listener at localhost.localdomain/45717] hbase.Waiter(180): Waiting up to [15,000] milli-secs(wait.for.ratio=[1]) 2023-07-13 06:00:44,633 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for testTablesInheritSnapshotSize5: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=2.25 MB/1 GB] 2023-07-13 06:00:44,633 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$14(433): Last observed size=-1 2023-07-13 06:00:45,136 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for testTablesInheritSnapshotSize5: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=2.25 MB/1 GB] 2023-07-13 06:00:45,137 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$14(433): Last observed size=2359562 2023-07-13 06:00:45,333 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x3f3506f5 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:45,336 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@65ca4715, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:45,338 DEBUG [hconnection-0xec71ef0-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:45,340 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:46714, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:45,640 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for testTablesInheritSnapshotSize5: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=2.25 MB/1 GB] 2023-07-13 06:00:45,640 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$14(433): Last observed size=2359562 2023-07-13 06:00:46,143 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for testTablesInheritSnapshotSize5: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=2.25 MB/1 GB] 2023-07-13 06:00:46,143 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$14(433): Last observed size=2359562 2023-07-13 06:00:46,300 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x4123abe1 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:46,304 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@14e26e1d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:46,306 DEBUG [hconnection-0x36fc6eba-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:46,309 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:46716, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:46,644 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for testTablesInheritSnapshotSize5: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=2.25 MB/1 GB] 2023-07-13 06:00:46,644 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$14(433): Last observed size=2359562 2023-07-13 06:00:47,146 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for testTablesInheritSnapshotSize5: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=2.25 MB/1 GB] 2023-07-13 06:00:47,146 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$14(433): Last observed size=2359562 2023-07-13 06:00:47,148 INFO [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots(135): Initial table size was 2359562 2023-07-13 06:00:47,148 INFO [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots(137): Snapshot the table 2023-07-13 06:00:47,149 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1583): Client=jenkins//148.251.75.209 snapshot request for:{ ss=testTablesInheritSnapshotSize5_snapshot1 table=testTablesInheritSnapshotSize5 type=FLUSH ttl=0 } 2023-07-13 06:00:47,149 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1689228047149 (current time:1689228047149). 2023-07-13 06:00:47,149 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-07-13 06:00:47,149 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] zookeeper.ReadOnlyZKClient(139): Connect 0x435dd711 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:47,153 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4f81044e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:47,155 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:47,157 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:46728, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:47,158 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x435dd711 to 127.0.0.1:55963 2023-07-13 06:00:47,158 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:47,158 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-07-13 06:00:47,159 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotManager(648): Table enabled, starting distributed snapshots for { ss=testTablesInheritSnapshotSize5_snapshot1 table=testTablesInheritSnapshotSize5 type=FLUSH ttl=0 } 2023-07-13 06:00:47,161 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testTablesInheritSnapshotSize5, type=EXCLUSIVE 2023-07-13 06:00:47,162 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=52, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testTablesInheritSnapshotSize5, type=EXCLUSIVE 2023-07-13 06:00:47,162 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=52, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testTablesInheritSnapshotSize5, type=EXCLUSIVE; timeout=600000, timestamp=1689228647162 2023-07-13 06:00:47,162 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.TakeSnapshotHandler(174): Running FLUSH table snapshot testTablesInheritSnapshotSize5_snapshot1 C_M_SNAPSHOT_TABLE on table testTablesInheritSnapshotSize5 2023-07-13 06:00:47,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotManager(653): Started snapshot: { ss=testTablesInheritSnapshotSize5_snapshot1 table=testTablesInheritSnapshotSize5 type=FLUSH ttl=0 } 2023-07-13 06:00:47,163 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=testTablesInheritSnapshotSize5_snapshot1 table=testTablesInheritSnapshotSize5 type=FLUSH ttl=0 }'' to complete. (max 20000 ms per retry) 2023-07-13 06:00:47,164 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testTablesInheritSnapshotSize5, type=SHARED 2023-07-13 06:00:47,164 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=52, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testTablesInheritSnapshotSize5, type=EXCLUSIVE 2023-07-13 06:00:47,164 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-07-13 06:00:47,165 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testTablesInheritSnapshotSize5, type=EXCLUSIVE in 4 msec 2023-07-13 06:00:47,165 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=53, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testTablesInheritSnapshotSize5, type=SHARED 2023-07-13 06:00:47,166 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=53, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testTablesInheritSnapshotSize5, type=SHARED; timeout=600000, timestamp=1689228647166 2023-07-13 06:00:47,259 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x7f371646 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:47,263 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@35d67577, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:47,264 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-07-13 06:00:47,265 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=testTablesInheritSnapshotSize5_snapshot1 table=testTablesInheritSnapshotSize5 type=FLUSH ttl=0 } is done 2023-07-13 06:00:47,266 DEBUG [hconnection-0x75e12554-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:47,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotManager(404): Snapshoting '{ ss=testTablesInheritSnapshotSize5_snapshot1 table=testTablesInheritSnapshotSize5 type=FLUSH ttl=0 }' is still in progress! 2023-07-13 06:00:47,266 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-07-13 06:00:47,269 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:46734, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:47,467 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-07-13 06:00:47,468 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=testTablesInheritSnapshotSize5_snapshot1 table=testTablesInheritSnapshotSize5 type=FLUSH ttl=0 } is done 2023-07-13 06:00:47,468 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotManager(404): Snapshoting '{ ss=testTablesInheritSnapshotSize5_snapshot1 table=testTablesInheritSnapshotSize5 type=FLUSH ttl=0 }' is still in progress! 2023-07-13 06:00:47,469 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-07-13 06:00:47,573 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] procedure.ProcedureCoordinator(165): Submitting procedure testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:47,573 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testTablesInheritSnapshotSize5_snapshot1' 2023-07-13 06:00:47,573 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-07-13 06:00:47,574 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testTablesInheritSnapshotSize5_snapshot1' starting 'acquire' 2023-07-13 06:00:47,574 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testTablesInheritSnapshotSize5_snapshot1', kicking off acquire phase on members. 2023-07-13 06:00:47,574 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:47,574 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/online-snapshot/acquired/testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:47,575 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/online-snapshot/acquired/testTablesInheritSnapshotSize5_snapshot1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:47,575 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-07-13 06:00:47,575 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-07-13 06:00:47,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-13 06:00:47,576 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/acquired/testTablesInheritSnapshotSize5_snapshot1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:47,576 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-13 06:00:47,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:47,576 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:47,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 91 2023-07-13 06:00:47,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:47,577 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot testTablesInheritSnapshotSize5_snapshot1 from table testTablesInheritSnapshotSize5 type FLUSH 2023-07-13 06:00:47,577 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:47,577 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testTablesInheritSnapshotSize5_snapshot1' with timeout 300000ms 2023-07-13 06:00:47,577 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-07-13 06:00:47,578 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testTablesInheritSnapshotSize5_snapshot1' starting 'acquire' stage 2023-07-13 06:00:47,578 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testTablesInheritSnapshotSize5_snapshot1' locally acquired 2023-07-13 06:00:47,578 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,35187,1689227983701' joining acquired barrier for procedure (testTablesInheritSnapshotSize5_snapshot1) in zk 2023-07-13 06:00:47,579 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/acquired/testTablesInheritSnapshotSize5_snapshot1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:47,579 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:47,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/acquired/testTablesInheritSnapshotSize5_snapshot1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:47,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:47,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-07-13 06:00:47,579 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:47,579 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testTablesInheritSnapshotSize5_snapshot1' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-13 06:00:47,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:47,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:47,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:47,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:47,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:47,581 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,35187,1689227983701' joining acquired barrier for procedure 'testTablesInheritSnapshotSize5_snapshot1' on coordinator 2023-07-13 06:00:47,581 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testTablesInheritSnapshotSize5_snapshot1' starting 'in-barrier' execution. 2023-07-13 06:00:47,581 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@27f7307c[Count = 0] remaining members to acquire global barrier 2023-07-13 06:00:47,581 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/online-snapshot/reached/testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:47,581 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:47,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:47,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:47,582 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testTablesInheritSnapshotSize5_snapshot1' received 'reached' from coordinator. 2023-07-13 06:00:47,582 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/testTablesInheritSnapshotSize5_snapshot1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:47,582 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] snapshot.FlushSnapshotSubprocedure(170): Flush Snapshot Tasks submitted for 1 regions 2023-07-13 06:00:47,582 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3. 2023-07-13 06:00:47,582 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-13 06:00:47,582 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3. started... 2023-07-13 06:00:47,582 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(301): Waiting for local region snapshots to finish. 2023-07-13 06:00:47,582 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] regionserver.HRegion(2446): Flush status journal for d8ad6096a0579ab248b74b5f53c0c5f3: 2023-07-13 06:00:47,582 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3.' region-info for snapshot=testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:47,582 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-07-13 06:00:47,583 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesInheritSnapshotSize5/d8ad6096a0579ab248b74b5f53c0c5f3/f1/209055e1a9db425fa87efcf218ca5d9d] hfiles 2023-07-13 06:00:47,583 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/1): hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesInheritSnapshotSize5/d8ad6096a0579ab248b74b5f53c0c5f3/f1/209055e1a9db425fa87efcf218ca5d9d for snapshot=testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:47,769 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-07-13 06:00:47,771 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=testTablesInheritSnapshotSize5_snapshot1 table=testTablesInheritSnapshotSize5 type=FLUSH ttl=0 } is done 2023-07-13 06:00:47,771 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotManager(404): Snapshoting '{ ss=testTablesInheritSnapshotSize5_snapshot1 table=testTablesInheritSnapshotSize5 type=FLUSH ttl=0 }' is still in progress! 2023-07-13 06:00:47,771 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-07-13 06:00:47,991 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3. completed. 2023-07-13 06:00:47,991 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3. 2023-07-13 06:00:47,991 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 1/1 local region snapshots. 2023-07-13 06:00:47,992 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(314): Completed 1 local region snapshots. 2023-07-13 06:00:47,992 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(345): cancelling 0 tasks for snapshot jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:47,992 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testTablesInheritSnapshotSize5_snapshot1' locally completed 2023-07-13 06:00:47,992 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testTablesInheritSnapshotSize5_snapshot1' completed for member 'jenkins-hbase20.apache.org,35187,1689227983701' in zk 2023-07-13 06:00:47,994 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/testTablesInheritSnapshotSize5_snapshot1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:47,994 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testTablesInheritSnapshotSize5_snapshot1' has notified controller of completion 2023-07-13 06:00:47,995 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-13 06:00:47,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/reached/testTablesInheritSnapshotSize5_snapshot1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:47,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:47,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-07-13 06:00:47,995 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testTablesInheritSnapshotSize5_snapshot1' completed. 2023-07-13 06:00:47,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:47,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:47,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:47,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:47,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:47,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:48,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:48,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testTablesInheritSnapshotSize5_snapshot1' member 'jenkins-hbase20.apache.org,35187,1689227983701': 2023-07-13 06:00:48,000 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,35187,1689227983701' released barrier for procedure'testTablesInheritSnapshotSize5_snapshot1', counting down latch. Waiting for 0 more 2023-07-13 06:00:48,000 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testTablesInheritSnapshotSize5_snapshot1' execution completed 2023-07-13 06:00:48,001 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-13 06:00:48,001 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-13 06:00:48,001 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:48,001 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testTablesInheritSnapshotSize5_snapshot1including nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2023-07-13 06:00:48,003 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:48,003 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:48,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:48,003 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-07-13 06:00:48,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:48,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/abort/testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:48,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:48,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-07-13 06:00:48,003 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/testTablesInheritSnapshotSize5_snapshot1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:48,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-07-13 06:00:48,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-13 06:00:48,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:48,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:48,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:48,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:48,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:48,005 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/testTablesInheritSnapshotSize5_snapshot1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:48,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:48,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:48,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:48,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:48,011 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/testTablesInheritSnapshotSize5_snapshot1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:48,011 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-07-13 06:00:48,011 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.EnabledTableSnapshotHandler(97): Done waiting - online snapshot for testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:48,011 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Unable to get data of znode /hbase/online-snapshot/abort/testTablesInheritSnapshotSize5_snapshot1 because node does not exist (not an error) 2023-07-13 06:00:48,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-07-13 06:00:48,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-13 06:00:48,011 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-07-13 06:00:48,011 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired/testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:48,011 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-13 06:00:48,011 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:48,011 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:48,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-07-13 06:00:48,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-13 06:00:48,012 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/testTablesInheritSnapshotSize5_snapshot1/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:48,012 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/reached/testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:48,012 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:48,012 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/abort/testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:48,012 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.hbase-snapshot/.tmp/testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:48,272 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-07-13 06:00:48,274 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=testTablesInheritSnapshotSize5_snapshot1 table=testTablesInheritSnapshotSize5 type=FLUSH ttl=0 } is done 2023-07-13 06:00:48,274 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotManager(404): Snapshoting '{ ss=testTablesInheritSnapshotSize5_snapshot1 table=testTablesInheritSnapshotSize5 type=FLUSH ttl=0 }' is still in progress! 2023-07-13 06:00:48,274 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-07-13 06:00:48,316 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x7d9d3f8d to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:48,318 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@16755125, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:48,320 DEBUG [hconnection-0x3de9f8c5-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:48,322 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:46748, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:48,426 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.hbase-snapshot/.tmp/testTablesInheritSnapshotSize5_snapshot1 to hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.hbase-snapshot/testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:48,565 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-13 06:00:49,240 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x4cbd4d68 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:49,242 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@56a4a3d9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:49,243 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.TakeSnapshotHandler(229): Snapshot testTablesInheritSnapshotSize5_snapshot1 of table testTablesInheritSnapshotSize5 completed 2023-07-13 06:00:49,243 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.hbase-snapshot/.tmp/testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:49,243 DEBUG [hconnection-0x5cbc2566-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:49,243 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/.hbase-snapshot/.tmp/testTablesInheritSnapshotSize5_snapshot1 2023-07-13 06:00:49,244 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running FLUSH table snapshot testTablesInheritSnapshotSize5_snapshot1 C_M_SNAPSHOT_TABLE on table testTablesInheritSnapshotSize5 at 1689228047162Consolidate snapshot: testTablesInheritSnapshotSize5_snapshot1 at 1689228048011 (+849 ms)Loading Region manifests for testTablesInheritSnapshotSize5_snapshot1 at 1689228048012 (+1 ms)Writing data manifest for testTablesInheritSnapshotSize5_snapshot1 at 1689228048014 (+2 ms)Verifying snapshot: testTablesInheritSnapshotSize5_snapshot1 at 1689228048421 (+407 ms)Snapshot testTablesInheritSnapshotSize5_snapshot1 of table testTablesInheritSnapshotSize5 completed at 1689228049243 (+822 ms) 2023-07-13 06:00:49,245 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=53, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testTablesInheritSnapshotSize5, type=SHARED 2023-07-13 06:00:49,245 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:52474, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:49,246 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testTablesInheritSnapshotSize5, type=SHARED in 2.0820 sec 2023-07-13 06:00:49,275 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-07-13 06:00:49,276 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=testTablesInheritSnapshotSize5_snapshot1 table=testTablesInheritSnapshotSize5 type=FLUSH ttl=0 } is done 2023-07-13 06:00:49,276 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] snapshot.SnapshotManager(401): Snapshot '{ ss=testTablesInheritSnapshotSize5_snapshot1 table=testTablesInheritSnapshotSize5 type=FLUSH ttl=0 }' has completed, notifying client. 2023-07-13 06:00:49,276 INFO [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots(143): Write more data 2023-07-13 06:00:49,464 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests(539): Data was written to HBase 2023-07-13 06:00:49,467 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-13 06:00:49,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] procedure.ProcedureCoordinator(143): Procedure testTablesInheritSnapshotSize5 was in running list but was completed. Accepting new attempt. 2023-07-13 06:00:49,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] procedure.ProcedureCoordinator(165): Submitting procedure testTablesInheritSnapshotSize5 2023-07-13 06:00:49,470 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testTablesInheritSnapshotSize5' 2023-07-13 06:00:49,470 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-13 06:00:49,471 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testTablesInheritSnapshotSize5' starting 'acquire' 2023-07-13 06:00:49,471 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testTablesInheritSnapshotSize5', kicking off acquire phase on members. 2023-07-13 06:00:49,471 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testTablesInheritSnapshotSize5 2023-07-13 06:00:49,472 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testTablesInheritSnapshotSize5 2023-07-13 06:00:49,473 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testTablesInheritSnapshotSize5/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:49,473 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-13 06:00:49,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-13 06:00:49,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-13 06:00:49,473 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testTablesInheritSnapshotSize5/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:49,473 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-13 06:00:49,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testTablesInheritSnapshotSize5 2023-07-13 06:00:49,475 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testTablesInheritSnapshotSize5 2023-07-13 06:00:49,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-13 06:00:49,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testTablesInheritSnapshotSize5 2023-07-13 06:00:49,475 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testTablesInheritSnapshotSize5 2023-07-13 06:00:49,476 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testTablesInheritSnapshotSize5 is still present, removing 2023-07-13 06:00:49,476 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testTablesInheritSnapshotSize5 2023-07-13 06:00:49,476 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testTablesInheritSnapshotSize5' with timeout 60000ms 2023-07-13 06:00:49,476 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-13 06:00:49,477 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testTablesInheritSnapshotSize5' starting 'acquire' stage 2023-07-13 06:00:49,477 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-13 06:00:49,477 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-13 06:00:49,477 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3. 2023-07-13 06:00:49,477 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3. started... 2023-07-13 06:00:49,477 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d8ad6096a0579ab248b74b5f53c0c5f3 1/1 column families, dataSize=2.22 MB heapSize=2.81 MB 2023-07-13 06:00:49,903 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.22 MB at sequenceid=330 (bloomFilter=true), to=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesInheritSnapshotSize5/d8ad6096a0579ab248b74b5f53c0c5f3/.tmp/f1/c5d3e95f5b0c46f096df81a33d6e6fcc 2023-07-13 06:00:49,909 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesInheritSnapshotSize5/d8ad6096a0579ab248b74b5f53c0c5f3/.tmp/f1/c5d3e95f5b0c46f096df81a33d6e6fcc as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesInheritSnapshotSize5/d8ad6096a0579ab248b74b5f53c0c5f3/f1/c5d3e95f5b0c46f096df81a33d6e6fcc 2023-07-13 06:00:49,915 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesInheritSnapshotSize5/d8ad6096a0579ab248b74b5f53c0c5f3/f1/c5d3e95f5b0c46f096df81a33d6e6fcc, entries=8192, sequenceid=330, filesize=2.3 M 2023-07-13 06:00:49,916 INFO [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.22 MB/2325418, heapSize ~2.81 MB/2949360, currentSize=0 B/0 for d8ad6096a0579ab248b74b5f53c0c5f3 in 439ms, sequenceid=330, compaction requested=false 2023-07-13 06:00:49,916 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d8ad6096a0579ab248b74b5f53c0c5f3: 2023-07-13 06:00:49,917 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3. 2023-07-13 06:00:49,917 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-13 06:00:49,917 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-13 06:00:49,917 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:49,917 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testTablesInheritSnapshotSize5' locally acquired 2023-07-13 06:00:49,917 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,35187,1689227983701' joining acquired barrier for procedure (testTablesInheritSnapshotSize5) in zk 2023-07-13 06:00:49,918 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testTablesInheritSnapshotSize5/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:49,918 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5 2023-07-13 06:00:49,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testTablesInheritSnapshotSize5/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:49,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:49,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:49,919 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5 2023-07-13 06:00:49,919 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testTablesInheritSnapshotSize5' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-13 06:00:49,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:49,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:49,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesInheritSnapshotSize5 2023-07-13 06:00:49,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:49,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:49,921 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,35187,1689227983701' joining acquired barrier for procedure 'testTablesInheritSnapshotSize5' on coordinator 2023-07-13 06:00:49,921 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@53de7d1[Count = 0] remaining members to acquire global barrier 2023-07-13 06:00:49,921 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testTablesInheritSnapshotSize5' starting 'in-barrier' execution. 2023-07-13 06:00:49,921 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5 2023-07-13 06:00:49,922 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5 2023-07-13 06:00:49,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5 2023-07-13 06:00:49,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5 2023-07-13 06:00:49,922 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testTablesInheritSnapshotSize5' received 'reached' from coordinator. 2023-07-13 06:00:49,922 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testTablesInheritSnapshotSize5' locally completed 2023-07-13 06:00:49,922 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testTablesInheritSnapshotSize5' completed for member 'jenkins-hbase20.apache.org,35187,1689227983701' in zk 2023-07-13 06:00:49,922 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:49,922 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-13 06:00:49,923 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:49,923 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testTablesInheritSnapshotSize5' has notified controller of completion 2023-07-13 06:00:49,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:49,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:49,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:49,923 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-13 06:00:49,923 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testTablesInheritSnapshotSize5' completed. 2023-07-13 06:00:49,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:49,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:49,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesInheritSnapshotSize5 2023-07-13 06:00:49,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:49,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:49,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesInheritSnapshotSize5 2023-07-13 06:00:49,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:49,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testTablesInheritSnapshotSize5' member 'jenkins-hbase20.apache.org,35187,1689227983701': 2023-07-13 06:00:49,926 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,35187,1689227983701' released barrier for procedure'testTablesInheritSnapshotSize5', counting down latch. Waiting for 0 more 2023-07-13 06:00:49,926 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testTablesInheritSnapshotSize5' execution completed 2023-07-13 06:00:49,926 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-13 06:00:49,926 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-13 06:00:49,926 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testTablesInheritSnapshotSize5 2023-07-13 06:00:49,926 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testTablesInheritSnapshotSize5including nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-13 06:00:49,927 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testTablesInheritSnapshotSize5 2023-07-13 06:00:49,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testTablesInheritSnapshotSize5 2023-07-13 06:00:49,927 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testTablesInheritSnapshotSize5 2023-07-13 06:00:49,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:49,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:49,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testTablesInheritSnapshotSize5 2023-07-13 06:00:49,928 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-13 06:00:49,928 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testTablesInheritSnapshotSize5/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:49,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testTablesInheritSnapshotSize5 2023-07-13 06:00:49,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:49,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-13 06:00:49,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesInheritSnapshotSize5 2023-07-13 06:00:49,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-13 06:00:49,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testTablesInheritSnapshotSize5 2023-07-13 06:00:49,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:49,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesInheritSnapshotSize5 2023-07-13 06:00:49,929 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:49,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:49,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:49,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesInheritSnapshotSize5 2023-07-13 06:00:49,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:49,932 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testTablesInheritSnapshotSize5/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:49,932 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-13 06:00:49,932 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testTablesInheritSnapshotSize5 2023-07-13 06:00:49,932 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-13 06:00:49,933 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testTablesInheritSnapshotSize5 because node does not exist (not an error) 2023-07-13 06:00:49,932 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testTablesInheritSnapshotSize5 2023-07-13 06:00:49,932 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testTablesInheritSnapshotSize5' 2023-07-13 06:00:49,933 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-13 06:00:49,932 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-13 06:00:49,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-13 06:00:49,932 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-13 06:00:49,933 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:49,934 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5 2023-07-13 06:00:49,934 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testTablesInheritSnapshotSize5'' to complete. (max 20000 ms per retry) 2023-07-13 06:00:49,934 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5 2023-07-13 06:00:49,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-13 06:00:49,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-13 06:00:49,934 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-13 06:00:49,934 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testTablesInheritSnapshotSize5 2023-07-13 06:00:50,034 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-13 06:00:50,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-13 06:00:50,035 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests(542): Data flushed to disk 2023-07-13 06:00:50,035 INFO [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots(145): Flush the table 2023-07-13 06:00:50,038 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(936): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-07-13 06:00:50,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] procedure.ProcedureCoordinator(143): Procedure testTablesInheritSnapshotSize5 was in running list but was completed. Accepting new attempt. 2023-07-13 06:00:50,039 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] procedure.ProcedureCoordinator(165): Submitting procedure testTablesInheritSnapshotSize5 2023-07-13 06:00:50,039 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testTablesInheritSnapshotSize5' 2023-07-13 06:00:50,039 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-13 06:00:50,040 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testTablesInheritSnapshotSize5' starting 'acquire' 2023-07-13 06:00:50,040 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testTablesInheritSnapshotSize5', kicking off acquire phase on members. 2023-07-13 06:00:50,040 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testTablesInheritSnapshotSize5 2023-07-13 06:00:50,040 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testTablesInheritSnapshotSize5 2023-07-13 06:00:50,041 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testTablesInheritSnapshotSize5/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:50,041 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-13 06:00:50,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-13 06:00:50,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-13 06:00:50,041 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testTablesInheritSnapshotSize5/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:50,041 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-13 06:00:50,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testTablesInheritSnapshotSize5 2023-07-13 06:00:50,042 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testTablesInheritSnapshotSize5 2023-07-13 06:00:50,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-13 06:00:50,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testTablesInheritSnapshotSize5 2023-07-13 06:00:50,042 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testTablesInheritSnapshotSize5 2023-07-13 06:00:50,042 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testTablesInheritSnapshotSize5 is still present, removing 2023-07-13 06:00:50,042 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testTablesInheritSnapshotSize5 2023-07-13 06:00:50,042 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testTablesInheritSnapshotSize5' with timeout 60000ms 2023-07-13 06:00:50,042 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-13 06:00:50,043 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testTablesInheritSnapshotSize5' starting 'acquire' stage 2023-07-13 06:00:50,043 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-13 06:00:50,043 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-13 06:00:50,043 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3. 2023-07-13 06:00:50,043 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3. started... 2023-07-13 06:00:50,043 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d8ad6096a0579ab248b74b5f53c0c5f3: 2023-07-13 06:00:50,043 DEBUG [rs(jenkins-hbase20.apache.org,35187,1689227983701)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3. 2023-07-13 06:00:50,043 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-13 06:00:50,044 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-13 06:00:50,044 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:50,044 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testTablesInheritSnapshotSize5' locally acquired 2023-07-13 06:00:50,044 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,35187,1689227983701' joining acquired barrier for procedure (testTablesInheritSnapshotSize5) in zk 2023-07-13 06:00:50,045 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testTablesInheritSnapshotSize5/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:50,045 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5 2023-07-13 06:00:50,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testTablesInheritSnapshotSize5/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:50,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:50,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:50,045 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5 2023-07-13 06:00:50,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:50,045 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testTablesInheritSnapshotSize5' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-13 06:00:50,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:50,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesInheritSnapshotSize5 2023-07-13 06:00:50,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:50,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:50,046 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,35187,1689227983701' joining acquired barrier for procedure 'testTablesInheritSnapshotSize5' on coordinator 2023-07-13 06:00:50,046 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6b8bccee[Count = 0] remaining members to acquire global barrier 2023-07-13 06:00:50,046 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testTablesInheritSnapshotSize5' starting 'in-barrier' execution. 2023-07-13 06:00:50,046 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5 2023-07-13 06:00:50,047 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5 2023-07-13 06:00:50,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5 2023-07-13 06:00:50,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5 2023-07-13 06:00:50,047 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testTablesInheritSnapshotSize5' received 'reached' from coordinator. 2023-07-13 06:00:50,047 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:50,047 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-13 06:00:50,047 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testTablesInheritSnapshotSize5' locally completed 2023-07-13 06:00:50,047 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testTablesInheritSnapshotSize5' completed for member 'jenkins-hbase20.apache.org,35187,1689227983701' in zk 2023-07-13 06:00:50,048 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:50,048 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testTablesInheritSnapshotSize5' has notified controller of completion 2023-07-13 06:00:50,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:50,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:50,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:50,048 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-13 06:00:50,048 DEBUG [member: 'jenkins-hbase20.apache.org,35187,1689227983701' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testTablesInheritSnapshotSize5' completed. 2023-07-13 06:00:50,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:50,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:50,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesInheritSnapshotSize5 2023-07-13 06:00:50,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:50,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:50,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesInheritSnapshotSize5 2023-07-13 06:00:50,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:50,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testTablesInheritSnapshotSize5' member 'jenkins-hbase20.apache.org,35187,1689227983701': 2023-07-13 06:00:50,051 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,35187,1689227983701' released barrier for procedure'testTablesInheritSnapshotSize5', counting down latch. Waiting for 0 more 2023-07-13 06:00:50,051 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testTablesInheritSnapshotSize5' execution completed 2023-07-13 06:00:50,051 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-13 06:00:50,051 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-13 06:00:50,051 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testTablesInheritSnapshotSize5 2023-07-13 06:00:50,051 INFO [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testTablesInheritSnapshotSize5including nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-13 06:00:50,052 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testTablesInheritSnapshotSize5 2023-07-13 06:00:50,052 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testTablesInheritSnapshotSize5 2023-07-13 06:00:50,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testTablesInheritSnapshotSize5 2023-07-13 06:00:50,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-13 06:00:50,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-13 06:00:50,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testTablesInheritSnapshotSize5 2023-07-13 06:00:50,052 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-13 06:00:50,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testTablesInheritSnapshotSize5 2023-07-13 06:00:50,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-13 06:00:50,052 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testTablesInheritSnapshotSize5/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:50,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-13 06:00:50,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-13 06:00:50,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesInheritSnapshotSize5 2023-07-13 06:00:50,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-13 06:00:50,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testTablesInheritSnapshotSize5 2023-07-13 06:00:50,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesInheritSnapshotSize5 2023-07-13 06:00:50,053 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:50,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:50,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-13 06:00:50,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testTablesInheritSnapshotSize5 2023-07-13 06:00:50,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:50,055 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testTablesInheritSnapshotSize5/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:50,055 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-13 06:00:50,055 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testTablesInheritSnapshotSize5 2023-07-13 06:00:50,055 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testTablesInheritSnapshotSize5' 2023-07-13 06:00:50,055 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-13 06:00:50,055 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-13 06:00:50,055 DEBUG [(jenkins-hbase20.apache.org,34737,1689227982756)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-13 06:00:50,055 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-13 06:00:50,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-13 06:00:50,055 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testTablesInheritSnapshotSize5 2023-07-13 06:00:50,056 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testTablesInheritSnapshotSize5'' to complete. (max 20000 ms per retry) 2023-07-13 06:00:50,055 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testTablesInheritSnapshotSize5 because node does not exist (not an error) 2023-07-13 06:00:50,056 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-13 06:00:50,056 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:50,056 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5 2023-07-13 06:00:50,056 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testTablesInheritSnapshotSize5 2023-07-13 06:00:50,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-13 06:00:50,056 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testTablesInheritSnapshotSize5 2023-07-13 06:00:50,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-13 06:00:50,156 DEBUG [Listener at localhost.localdomain/45717] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-13 06:00:50,157 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34737] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-13 06:00:50,158 INFO [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots(147): Synchronously compacting the table 2023-07-13 06:00:50,158 DEBUG [Listener at localhost.localdomain/45717] compactions.SortedCompactionPolicy(75): Selecting compaction from 2 store files, 0 compacting, 2 eligible, 16 blocking 2023-07-13 06:00:50,158 DEBUG [Listener at localhost.localdomain/45717] regionserver.HStore(1912): d8ad6096a0579ab248b74b5f53c0c5f3/f1 is initiating major compaction (all files) 2023-07-13 06:00:50,158 INFO [Listener at localhost.localdomain/45717] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-13 06:00:50,159 INFO [Listener at localhost.localdomain/45717] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-13 06:00:50,159 INFO [Listener at localhost.localdomain/45717] regionserver.HRegion(2259): Starting compaction of d8ad6096a0579ab248b74b5f53c0c5f3/f1 in testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3. 2023-07-13 06:00:50,159 INFO [Listener at localhost.localdomain/45717] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesInheritSnapshotSize5/d8ad6096a0579ab248b74b5f53c0c5f3/f1/209055e1a9db425fa87efcf218ca5d9d, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesInheritSnapshotSize5/d8ad6096a0579ab248b74b5f53c0c5f3/f1/c5d3e95f5b0c46f096df81a33d6e6fcc] into tmpdir=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesInheritSnapshotSize5/d8ad6096a0579ab248b74b5f53c0c5f3/.tmp, totalSize=4.5 M 2023-07-13 06:00:50,160 DEBUG [Listener at localhost.localdomain/45717] compactions.Compactor(207): Compacting 209055e1a9db425fa87efcf218ca5d9d, keycount=8192, bloomtype=ROW, size=2.3 M, encoding=NONE, compression=NONE, seqNum=165, earliestPutTs=1689228043393 2023-07-13 06:00:50,160 DEBUG [Listener at localhost.localdomain/45717] compactions.Compactor(207): Compacting c5d3e95f5b0c46f096df81a33d6e6fcc, keycount=8192, bloomtype=ROW, size=2.3 M, encoding=NONE, compression=NONE, seqNum=330, earliestPutTs=1689228049277 2023-07-13 06:00:50,206 INFO [Listener at localhost.localdomain/45717] throttle.PressureAwareThroughputController(145): d8ad6096a0579ab248b74b5f53c0c5f3#f1#compaction#35 average throughput is 58.36 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-13 06:00:50,226 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x2626388e to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:50,228 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7e695d48, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:50,230 DEBUG [hconnection-0x3a55a7c3-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:50,232 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:52486, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:50,276 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x541becf1 to 127.0.0.1:55963 2023-07-13 06:00:50,276 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:50,277 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7d9d3f8d to 127.0.0.1:55963 2023-07-13 06:00:50,277 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:50,277 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4cbd4d68 to 127.0.0.1:55963 2023-07-13 06:00:50,277 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:50,278 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2626388e to 127.0.0.1:55963 2023-07-13 06:00:50,278 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:50,278 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2659fe5b to 127.0.0.1:55963 2023-07-13 06:00:50,278 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:50,278 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3f3506f5 to 127.0.0.1:55963 2023-07-13 06:00:50,278 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:50,278 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4123abe1 to 127.0.0.1:55963 2023-07-13 06:00:50,278 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:50,278 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7f371646 to 127.0.0.1:55963 2023-07-13 06:00:50,278 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:50,278 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x790afb59 to 127.0.0.1:55963 2023-07-13 06:00:50,278 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:50,278 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x68c2fdd6 to 127.0.0.1:55963 2023-07-13 06:00:50,278 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:50,278 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1a274eac to 127.0.0.1:55963 2023-07-13 06:00:50,279 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:50,279 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x07578a92 to 127.0.0.1:55963 2023-07-13 06:00:50,279 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:50,279 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0fbfe2ca to 127.0.0.1:55963 2023-07-13 06:00:50,279 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:50,279 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x60fc217a to 127.0.0.1:55963 2023-07-13 06:00:50,279 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:50,279 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x16604c10 to 127.0.0.1:55963 2023-07-13 06:00:50,279 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:50,279 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4769ecd3 to 127.0.0.1:55963 2023-07-13 06:00:50,279 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:50,279 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x58f2bc25 to 127.0.0.1:55963 2023-07-13 06:00:50,279 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:50,279 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5aa82376 to 127.0.0.1:55963 2023-07-13 06:00:50,279 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:50,362 DEBUG [master/jenkins-hbase20:0.Chore.1] quotas.QuotaObserverChore(336): testTablesInheritSnapshotSize5 moved into observance of table space quota. 2023-07-13 06:00:50,617 DEBUG [Listener at localhost.localdomain/45717] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesInheritSnapshotSize5/d8ad6096a0579ab248b74b5f53c0c5f3/.tmp/f1/80e1735d113b4ebfac871f0553d42565 as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesInheritSnapshotSize5/d8ad6096a0579ab248b74b5f53c0c5f3/f1/80e1735d113b4ebfac871f0553d42565 2023-07-13 06:00:50,623 INFO [Listener at localhost.localdomain/45717] regionserver.HStore(1652): Completed major compaction of 2 (all) file(s) in d8ad6096a0579ab248b74b5f53c0c5f3/f1 of d8ad6096a0579ab248b74b5f53c0c5f3 into 80e1735d113b4ebfac871f0553d42565(size=2.3 M), total size for store is 2.3 M. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-13 06:00:50,623 DEBUG [Listener at localhost.localdomain/45717] regionserver.HRegion(2289): Compaction status journal for d8ad6096a0579ab248b74b5f53c0c5f3: 2023-07-13 06:00:50,624 INFO [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots(154): Waiting for the region reports to reflect the correct size, between (1585152, 2609152) 2023-07-13 06:00:50,624 INFO [Listener at localhost.localdomain/45717] hbase.Waiter(180): Waiting up to [30,000] milli-secs(wait.for.ratio=[1]) 2023-07-13 06:00:51,295 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x2495f85e to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:51,298 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@69449fae, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:51,300 DEBUG [hconnection-0x15a5e2bf-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:51,302 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:52494, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:51,631 INFO [Listener at localhost.localdomain/45717] hbase.Waiter(180): Waiting up to [15,000] milli-secs(wait.for.ratio=[1]) 2023-07-13 06:00:51,631 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$15(466): Last observed size=-1 2023-07-13 06:00:52,133 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$15(466): Last observed size=2369718 2023-07-13 06:00:52,266 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x20d3db4b to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:52,272 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6b5a3659, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:52,274 DEBUG [hconnection-0x27089777-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:52,276 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:52502, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:52,393 DEBUG [master/jenkins-hbase20:0.Chore.1] quotas.QuotaObserverChore(336): testTablesInheritSnapshotSize5 moved into observance of table space quota. 2023-07-13 06:00:52,637 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$15(466): Last observed size=2369718 2023-07-13 06:00:53,140 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$15(466): Last observed size=2369718 2023-07-13 06:00:53,225 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x57445309 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:53,230 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4b52f87a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:53,233 DEBUG [hconnection-0x63328fd9-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:53,236 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:52516, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:53,643 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$15(466): Last observed size=2369718 2023-07-13 06:00:54,145 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$15(466): Last observed size=2369718 2023-07-13 06:00:54,148 INFO [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots(168): Last seen size: 2369718 2023-07-13 06:00:54,149 INFO [Listener at localhost.localdomain/45717] hbase.Waiter(180): Waiting up to [20,000] milli-secs(wait.for.ratio=[1]) 2023-07-13 06:00:54,151 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for testTablesInheritSnapshotSize5: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=2.26 MB/1 GB] 2023-07-13 06:00:54,151 INFO [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots(183): Expecting table usage to be 2359562 + 2369718 = 4729280 2023-07-13 06:00:54,152 INFO [Listener at localhost.localdomain/45717] hbase.Waiter(180): Waiting up to [30,000] milli-secs(wait.for.ratio=[1]) 2023-07-13 06:00:54,153 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for testTablesInheritSnapshotSize5: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=2.26 MB/1 GB] 2023-07-13 06:00:54,154 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$5(189): Checking for 4729280 == 2369718 2023-07-13 06:00:54,238 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x30bca37c to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:54,241 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4465582b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:54,242 DEBUG [hconnection-0x29943165-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:54,245 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:52528, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:55,157 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for testTablesInheritSnapshotSize5: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=2.26 MB/1 GB] 2023-07-13 06:00:55,157 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$5(189): Checking for 4729280 == 2369718 2023-07-13 06:00:55,242 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x5c8010cf to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:55,245 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@62bc4707, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:55,247 DEBUG [hconnection-0x7144aee2-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:55,247 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:52532, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:55,299 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesInheritSnapshotSize5/d8ad6096a0579ab248b74b5f53c0c5f3/f1/209055e1a9db425fa87efcf218ca5d9d, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesInheritSnapshotSize5/d8ad6096a0579ab248b74b5f53c0c5f3/f1/c5d3e95f5b0c46f096df81a33d6e6fcc] to archive 2023-07-13 06:00:55,300 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-07-13 06:00:55,302 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesInheritSnapshotSize5/d8ad6096a0579ab248b74b5f53c0c5f3/f1/209055e1a9db425fa87efcf218ca5d9d to hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/archive/data/default/testTablesInheritSnapshotSize5/d8ad6096a0579ab248b74b5f53c0c5f3/f1/209055e1a9db425fa87efcf218ca5d9d 2023-07-13 06:00:55,303 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesInheritSnapshotSize5/d8ad6096a0579ab248b74b5f53c0c5f3/f1/c5d3e95f5b0c46f096df81a33d6e6fcc to hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/archive/data/default/testTablesInheritSnapshotSize5/d8ad6096a0579ab248b74b5f53c0c5f3/f1/c5d3e95f5b0c46f096df81a33d6e6fcc 2023-07-13 06:00:56,159 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for testTablesInheritSnapshotSize5: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=2.26 MB/1 GB] 2023-07-13 06:00:56,160 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$5(189): Checking for 4729280 == 2369718 2023-07-13 06:00:56,245 DEBUG [master/jenkins-hbase20:0.Chore.1] quotas.QuotaObserverChore(336): testTablesInheritSnapshotSize5 moved into observance of table space quota. 2023-07-13 06:00:56,246 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x46c45555 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:56,249 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1e29981b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:56,250 DEBUG [hconnection-0x2964dc59-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:56,255 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:52548, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:57,162 DEBUG [Listener at localhost.localdomain/45717] quotas.SpaceQuotaHelperForTests$SpaceQuotaSnapshotPredicate(731): Saw quota snapshot for testTablesInheritSnapshotSize5: SpaceQuotaSnapshot[policy=SpaceQuotaStatus[policy=Optional.empty, inViolation=false], use=4.51 MB/1 GB] 2023-07-13 06:00:57,162 DEBUG [Listener at localhost.localdomain/45717] quotas.TestSpaceQuotasWithSnapshots$5(189): Checking for 4729280 == 4729280 2023-07-13 06:00:57,184 INFO [Listener at localhost.localdomain/45717] hbase.ResourceChecker(175): after: quotas.TestSpaceQuotasWithSnapshots#testTablesInheritSnapshotSize Thread=234 (was 263), OpenFileDescriptor=524 (was 611), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=114 (was 128), ProcessCount=166 (was 168), AvailableMemoryMB=6389 (was 6528) 2023-07-13 06:00:57,185 INFO [Listener at localhost.localdomain/45717] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-07-13 06:00:57,185 INFO [Listener at localhost.localdomain/45717] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-07-13 06:00:57,185 DEBUG [Listener at localhost.localdomain/45717] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x404a8258 to 127.0.0.1:55963 2023-07-13 06:00:57,185 DEBUG [Listener at localhost.localdomain/45717] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:57,185 DEBUG [Listener at localhost.localdomain/45717] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-13 06:00:57,186 DEBUG [Listener at localhost.localdomain/45717] util.JVMClusterUtil(257): Found active master hash=732200442, stopped=false 2023-07-13 06:00:57,186 DEBUG [Listener at localhost.localdomain/45717] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.quotas.MasterQuotasObserver 2023-07-13 06:00:57,187 INFO [Listener at localhost.localdomain/45717] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase20.apache.org,34737,1689227982756 2023-07-13 06:00:57,188 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-13 06:00:57,188 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-13 06:00:57,188 INFO [Listener at localhost.localdomain/45717] procedure2.ProcedureExecutor(629): Stopping 2023-07-13 06:00:57,188 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-13 06:00:57,189 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-13 06:00:57,189 DEBUG [Listener at localhost.localdomain/45717] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x668f607f to 127.0.0.1:55963 2023-07-13 06:00:57,189 DEBUG [Listener at localhost.localdomain/45717] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:57,189 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-13 06:00:57,189 INFO [Listener at localhost.localdomain/45717] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase20.apache.org,35187,1689227983701' ***** 2023-07-13 06:00:57,189 INFO [Listener at localhost.localdomain/45717] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-13 06:00:57,189 INFO [RS:0;jenkins-hbase20:35187] regionserver.HeapMemoryManager(220): Stopping 2023-07-13 06:00:57,190 INFO [RS:0;jenkins-hbase20:35187] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-13 06:00:57,190 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-13 06:00:57,190 INFO [RS:0;jenkins-hbase20:35187] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-13 06:00:57,190 INFO [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(3305): Received CLOSE for c96fda412d307d645bd66b267f4787dd 2023-07-13 06:00:57,191 INFO [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(3305): Received CLOSE for d8ad6096a0579ab248b74b5f53c0c5f3 2023-07-13 06:00:57,192 INFO [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(3305): Received CLOSE for 2994958daca2f80926fd1799582c2b37 2023-07-13 06:00:57,192 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing c96fda412d307d645bd66b267f4787dd, disabling compactions & flushes 2023-07-13 06:00:57,192 INFO [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(3305): Received CLOSE for 71f62f4fe1b8de992835ee80c072a8c8 2023-07-13 06:00:57,192 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd. 2023-07-13 06:00:57,192 INFO [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(3305): Received CLOSE for d0f4e6ea3f932d1de6f1c8e1ab57b30a 2023-07-13 06:00:57,192 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd. 2023-07-13 06:00:57,192 INFO [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(3305): Received CLOSE for 6817724a8c0ba54dffa0384c0e1b4834 2023-07-13 06:00:57,192 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd. after waiting 0 ms 2023-07-13 06:00:57,192 INFO [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(3305): Received CLOSE for 41c84bc63b9629db7bd4a197ebc93b69 2023-07-13 06:00:57,192 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd. 2023-07-13 06:00:57,192 INFO [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(3305): Received CLOSE for 0d24aa4917843c73c4cba790560f2b8d 2023-07-13 06:00:57,192 INFO [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(3305): Received CLOSE for 186edc75693ac04a787ff0d8059708f3 2023-07-13 06:00:57,192 INFO [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(3305): Received CLOSE for ea36c09b963764399342ace600417093 2023-07-13 06:00:57,192 INFO [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(3305): Received CLOSE for b6e943f7ce0da690354636beb379d14f 2023-07-13 06:00:57,192 INFO [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(3305): Received CLOSE for 87846b634b85323bd205e28cff8a7ccc 2023-07-13 06:00:57,192 INFO [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(3305): Received CLOSE for 777f666dd9b789f8aac944c77f179207 2023-07-13 06:00:57,192 INFO [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(3305): Received CLOSE for c8571aee40cfd11e8cbf1a696291b517 2023-07-13 06:00:57,193 INFO [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(3305): Received CLOSE for b06e3484b8c05bbdc046fc6b5386dddc 2023-07-13 06:00:57,193 INFO [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(3305): Received CLOSE for e16a1a1ea32dc6c137274851c98ece04 2023-07-13 06:00:57,193 INFO [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(1144): stopping server jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:57,193 INFO [RS:0;jenkins-hbase20:35187] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-07-13 06:00:57,193 DEBUG [RS:0;jenkins-hbase20:35187] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x288e5e7b to 127.0.0.1:55963 2023-07-13 06:00:57,193 DEBUG [RS:0;jenkins-hbase20:35187] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:57,193 INFO [RS:0;jenkins-hbase20:35187] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-13 06:00:57,193 INFO [RS:0;jenkins-hbase20:35187] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-13 06:00:57,193 INFO [RS:0;jenkins-hbase20:35187] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-13 06:00:57,193 INFO [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-13 06:00:57,194 INFO [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(1474): Waiting on 17 regions to close 2023-07-13 06:00:57,194 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-13 06:00:57,195 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-13 06:00:57,195 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-13 06:00:57,195 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-13 06:00:57,195 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-13 06:00:57,195 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=22.23 KB heapSize=35.76 KB 2023-07-13 06:00:57,200 DEBUG [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(1504): Waiting on 0d24aa4917843c73c4cba790560f2b8d, 1588230740, 186edc75693ac04a787ff0d8059708f3, 2994958daca2f80926fd1799582c2b37, 41c84bc63b9629db7bd4a197ebc93b69, 6817724a8c0ba54dffa0384c0e1b4834, 71f62f4fe1b8de992835ee80c072a8c8, 777f666dd9b789f8aac944c77f179207, 87846b634b85323bd205e28cff8a7ccc, b06e3484b8c05bbdc046fc6b5386dddc, b6e943f7ce0da690354636beb379d14f, c8571aee40cfd11e8cbf1a696291b517, c96fda412d307d645bd66b267f4787dd, d0f4e6ea3f932d1de6f1c8e1ab57b30a, d8ad6096a0579ab248b74b5f53c0c5f3, e16a1a1ea32dc6c137274851c98ece04, ea36c09b963764399342ace600417093 2023-07-13 06:00:57,201 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testRematerializedTablesDoNoInheritSpace0/c96fda412d307d645bd66b267f4787dd/recovered.edits/334.seqid, newMaxSeqId=334, maxSeqId=1 2023-07-13 06:00:57,202 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd. 2023-07-13 06:00:57,202 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for c96fda412d307d645bd66b267f4787dd: 2023-07-13 06:00:57,203 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed testRematerializedTablesDoNoInheritSpace0,,1689227989151.c96fda412d307d645bd66b267f4787dd. 2023-07-13 06:00:57,203 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing d8ad6096a0579ab248b74b5f53c0c5f3, disabling compactions & flushes 2023-07-13 06:00:57,203 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3. 2023-07-13 06:00:57,203 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3. 2023-07-13 06:00:57,203 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3. after waiting 0 ms 2023-07-13 06:00:57,203 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3. 2023-07-13 06:00:57,208 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesInheritSnapshotSize5/d8ad6096a0579ab248b74b5f53c0c5f3/recovered.edits/335.seqid, newMaxSeqId=335, maxSeqId=1 2023-07-13 06:00:57,210 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3. 2023-07-13 06:00:57,210 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for d8ad6096a0579ab248b74b5f53c0c5f3: 2023-07-13 06:00:57,210 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed testTablesInheritSnapshotSize5,,1689228042280.d8ad6096a0579ab248b74b5f53c0c5f3. 2023-07-13 06:00:57,210 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 2994958daca2f80926fd1799582c2b37, disabling compactions & flushes 2023-07-13 06:00:57,210 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37. 2023-07-13 06:00:57,210 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37. 2023-07-13 06:00:57,210 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37. after waiting 0 ms 2023-07-13 06:00:57,210 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37. 2023-07-13 06:00:57,227 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=21.40 KB at sequenceid=57 (bloomFilter=false), to=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/meta/1588230740/.tmp/info/7952706754f64aeea82db97ba19a3639 2023-07-13 06:00:57,230 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/2994958daca2f80926fd1799582c2b37/recovered.edits/175.seqid, newMaxSeqId=175, maxSeqId=1 2023-07-13 06:00:57,232 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37. 2023-07-13 06:00:57,232 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 2994958daca2f80926fd1799582c2b37: 2023-07-13 06:00:57,232 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed testTablesWithSnapshots4,3,1689228022139.2994958daca2f80926fd1799582c2b37. 2023-07-13 06:00:57,232 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 71f62f4fe1b8de992835ee80c072a8c8, disabling compactions & flushes 2023-07-13 06:00:57,232 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8. 2023-07-13 06:00:57,232 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8. 2023-07-13 06:00:57,232 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8. after waiting 0 ms 2023-07-13 06:00:57,232 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8. 2023-07-13 06:00:57,237 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/71f62f4fe1b8de992835ee80c072a8c8/recovered.edits/8.seqid, newMaxSeqId=8, maxSeqId=1 2023-07-13 06:00:57,239 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8. 2023-07-13 06:00:57,239 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 71f62f4fe1b8de992835ee80c072a8c8: 2023-07-13 06:00:57,239 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed testTablesWithSnapshots4,,1689228022139.71f62f4fe1b8de992835ee80c072a8c8. 2023-07-13 06:00:57,240 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing d0f4e6ea3f932d1de6f1c8e1ab57b30a, disabling compactions & flushes 2023-07-13 06:00:57,240 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1689227986607.d0f4e6ea3f932d1de6f1c8e1ab57b30a. 2023-07-13 06:00:57,240 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1689227986607.d0f4e6ea3f932d1de6f1c8e1ab57b30a. 2023-07-13 06:00:57,240 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1689227986607.d0f4e6ea3f932d1de6f1c8e1ab57b30a. after waiting 0 ms 2023-07-13 06:00:57,240 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1689227986607.d0f4e6ea3f932d1de6f1c8e1ab57b30a. 2023-07-13 06:00:57,240 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2745): Flushing d0f4e6ea3f932d1de6f1c8e1ab57b30a 1/1 column families, dataSize=111 B heapSize=600 B 2023-07-13 06:00:57,242 INFO [regionserver/jenkins-hbase20:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-13 06:00:57,256 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=111 B at sequenceid=7 (bloomFilter=true), to=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/namespace/d0f4e6ea3f932d1de6f1c8e1ab57b30a/.tmp/info/e9ff237d6c8f4a5ca1cf59137c9d92de 2023-07-13 06:00:57,260 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/namespace/d0f4e6ea3f932d1de6f1c8e1ab57b30a/.tmp/info/e9ff237d6c8f4a5ca1cf59137c9d92de as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/namespace/d0f4e6ea3f932d1de6f1c8e1ab57b30a/info/e9ff237d6c8f4a5ca1cf59137c9d92de 2023-07-13 06:00:57,261 INFO [regionserver/jenkins-hbase20:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-07-13 06:00:57,261 INFO [regionserver/jenkins-hbase20:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-07-13 06:00:57,261 INFO [regionserver/jenkins-hbase20:0.Chore.1] hbase.ScheduledChore(146): Chore: FileSystemUtilizationChore was stopped 2023-07-13 06:00:57,263 DEBUG [master/jenkins-hbase20:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x285e2c13 to 127.0.0.1:55963 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-13 06:00:57,264 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/namespace/d0f4e6ea3f932d1de6f1c8e1ab57b30a/info/e9ff237d6c8f4a5ca1cf59137c9d92de, entries=3, sequenceid=7, filesize=4.8 K 2023-07-13 06:00:57,265 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~111 B/111, heapSize ~584 B/584, currentSize=0 B/0 for d0f4e6ea3f932d1de6f1c8e1ab57b30a in 25ms, sequenceid=7, compaction requested=false 2023-07-13 06:00:57,268 DEBUG [master/jenkins-hbase20:0.Chore.1] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@60330120, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-13 06:00:57,269 DEBUG [hconnection-0xe2c3979-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-13 06:00:57,270 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/namespace/d0f4e6ea3f932d1de6f1c8e1ab57b30a/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-07-13 06:00:57,272 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:52550, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-13 06:00:57,273 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1689227986607.d0f4e6ea3f932d1de6f1c8e1ab57b30a. 2023-07-13 06:00:57,273 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for d0f4e6ea3f932d1de6f1c8e1ab57b30a: 2023-07-13 06:00:57,273 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1689227986607.d0f4e6ea3f932d1de6f1c8e1ab57b30a. 2023-07-13 06:00:57,273 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=35187] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.regionserver.RegionServerStoppedException: Server jenkins-hbase20.apache.org,35187,1689227983701 stopping at org.apache.hadoop.hbase.regionserver.RSRpcServices.checkOpen(RSRpcServices.java:1527) at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3494) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44996) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-13 06:00:57,273 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 6817724a8c0ba54dffa0384c0e1b4834, disabling compactions & flushes 2023-07-13 06:00:57,273 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834. 2023-07-13 06:00:57,273 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834. 2023-07-13 06:00:57,273 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834. after waiting 0 ms 2023-07-13 06:00:57,273 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834. 2023-07-13 06:00:57,273 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=35187] ipc.CallRunner(144): callId: 0 service: ClientService methodName: Scan size: 131 connection: 148.251.75.209:52550 deadline: 1689228117273, exception=org.apache.hadoop.hbase.regionserver.RegionServerStoppedException: Server jenkins-hbase20.apache.org,35187,1689227983701 stopping 2023-07-13 06:00:57,277 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/6817724a8c0ba54dffa0384c0e1b4834/recovered.edits/175.seqid, newMaxSeqId=175, maxSeqId=1 2023-07-13 06:00:57,278 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834. 2023-07-13 06:00:57,278 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 6817724a8c0ba54dffa0384c0e1b4834: 2023-07-13 06:00:57,278 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed testTablesWithSnapshots4,9,1689228022139.6817724a8c0ba54dffa0384c0e1b4834. 2023-07-13 06:00:57,278 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 41c84bc63b9629db7bd4a197ebc93b69, disabling compactions & flushes 2023-07-13 06:00:57,278 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69. 2023-07-13 06:00:57,278 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69. 2023-07-13 06:00:57,278 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69. after waiting 0 ms 2023-07-13 06:00:57,278 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69. 2023-07-13 06:00:57,281 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/41c84bc63b9629db7bd4a197ebc93b69/recovered.edits/175.seqid, newMaxSeqId=175, maxSeqId=1 2023-07-13 06:00:57,282 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69. 2023-07-13 06:00:57,282 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 41c84bc63b9629db7bd4a197ebc93b69: 2023-07-13 06:00:57,282 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed testTablesWithSnapshots4,1,1689228022139.41c84bc63b9629db7bd4a197ebc93b69. 2023-07-13 06:00:57,282 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 0d24aa4917843c73c4cba790560f2b8d, disabling compactions & flushes 2023-07-13 06:00:57,282 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d. 2023-07-13 06:00:57,282 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d. 2023-07-13 06:00:57,282 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d. after waiting 0 ms 2023-07-13 06:00:57,282 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d. 2023-07-13 06:00:57,286 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/0d24aa4917843c73c4cba790560f2b8d/recovered.edits/175.seqid, newMaxSeqId=175, maxSeqId=1 2023-07-13 06:00:57,286 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d. 2023-07-13 06:00:57,287 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 0d24aa4917843c73c4cba790560f2b8d: 2023-07-13 06:00:57,287 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed testTablesWithSnapshots4,4,1689228022139.0d24aa4917843c73c4cba790560f2b8d. 2023-07-13 06:00:57,287 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 186edc75693ac04a787ff0d8059708f3, disabling compactions & flushes 2023-07-13 06:00:57,287 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3. 2023-07-13 06:00:57,287 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3. 2023-07-13 06:00:57,287 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3. after waiting 0 ms 2023-07-13 06:00:57,287 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3. 2023-07-13 06:00:57,290 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/186edc75693ac04a787ff0d8059708f3/recovered.edits/175.seqid, newMaxSeqId=175, maxSeqId=1 2023-07-13 06:00:57,291 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3. 2023-07-13 06:00:57,291 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 186edc75693ac04a787ff0d8059708f3: 2023-07-13 06:00:57,291 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed testTablesWithSnapshots4,7,1689228022139.186edc75693ac04a787ff0d8059708f3. 2023-07-13 06:00:57,291 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing ea36c09b963764399342ace600417093, disabling compactions & flushes 2023-07-13 06:00:57,291 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093. 2023-07-13 06:00:57,291 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093. 2023-07-13 06:00:57,291 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093. after waiting 0 ms 2023-07-13 06:00:57,292 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093. 2023-07-13 06:00:57,293 INFO [regionserver/jenkins-hbase20:0.Chore.1] hbase.ScheduledChore(146): Chore: RegionSizeReportingChore was stopped 2023-07-13 06:00:57,293 INFO [regionserver/jenkins-hbase20:0.Chore.1] hbase.ScheduledChore(146): Chore: SpaceQuotaRefresherChore was stopped 2023-07-13 06:00:57,294 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/ea36c09b963764399342ace600417093/recovered.edits/175.seqid, newMaxSeqId=175, maxSeqId=1 2023-07-13 06:00:57,295 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093. 2023-07-13 06:00:57,295 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for ea36c09b963764399342ace600417093: 2023-07-13 06:00:57,296 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed testTablesWithSnapshots4,2,1689228022139.ea36c09b963764399342ace600417093. 2023-07-13 06:00:57,296 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing b6e943f7ce0da690354636beb379d14f, disabling compactions & flushes 2023-07-13 06:00:57,296 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f. 2023-07-13 06:00:57,296 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f. 2023-07-13 06:00:57,296 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f. after waiting 0 ms 2023-07-13 06:00:57,296 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f. 2023-07-13 06:00:57,299 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/ns2/testNamespacesInheritSnapshotSize3/b6e943f7ce0da690354636beb379d14f/recovered.edits/336.seqid, newMaxSeqId=336, maxSeqId=1 2023-07-13 06:00:57,300 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f. 2023-07-13 06:00:57,300 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for b6e943f7ce0da690354636beb379d14f: 2023-07-13 06:00:57,300 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed ns2:testNamespacesInheritSnapshotSize3,,1689228003732.b6e943f7ce0da690354636beb379d14f. 2023-07-13 06:00:57,300 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 87846b634b85323bd205e28cff8a7ccc, disabling compactions & flushes 2023-07-13 06:00:57,300 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc. 2023-07-13 06:00:57,300 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc. 2023-07-13 06:00:57,300 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc. after waiting 0 ms 2023-07-13 06:00:57,300 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc. 2023-07-13 06:00:57,303 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/87846b634b85323bd205e28cff8a7ccc/recovered.edits/175.seqid, newMaxSeqId=175, maxSeqId=1 2023-07-13 06:00:57,304 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc. 2023-07-13 06:00:57,304 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 87846b634b85323bd205e28cff8a7ccc: 2023-07-13 06:00:57,304 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed testTablesWithSnapshots4,6,1689228022139.87846b634b85323bd205e28cff8a7ccc. 2023-07-13 06:00:57,304 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 777f666dd9b789f8aac944c77f179207, disabling compactions & flushes 2023-07-13 06:00:57,305 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testRematerializedTablesDoNoInheritSpace1,,1689227989151.777f666dd9b789f8aac944c77f179207. 2023-07-13 06:00:57,305 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRematerializedTablesDoNoInheritSpace1,,1689227989151.777f666dd9b789f8aac944c77f179207. 2023-07-13 06:00:57,305 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testRematerializedTablesDoNoInheritSpace1,,1689227989151.777f666dd9b789f8aac944c77f179207. after waiting 0 ms 2023-07-13 06:00:57,305 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testRematerializedTablesDoNoInheritSpace1,,1689227989151.777f666dd9b789f8aac944c77f179207. 2023-07-13 06:00:57,307 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testRematerializedTablesDoNoInheritSpace1/777f666dd9b789f8aac944c77f179207/recovered.edits/169.seqid, newMaxSeqId=169, maxSeqId=165 2023-07-13 06:00:57,308 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testRematerializedTablesDoNoInheritSpace1,,1689227989151.777f666dd9b789f8aac944c77f179207. 2023-07-13 06:00:57,308 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 777f666dd9b789f8aac944c77f179207: 2023-07-13 06:00:57,309 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed testRematerializedTablesDoNoInheritSpace1,,1689227989151.777f666dd9b789f8aac944c77f179207. 2023-07-13 06:00:57,309 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing c8571aee40cfd11e8cbf1a696291b517, disabling compactions & flushes 2023-07-13 06:00:57,309 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region hbase:quota,,1689227988210.c8571aee40cfd11e8cbf1a696291b517. 2023-07-13 06:00:57,309 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:quota,,1689227988210.c8571aee40cfd11e8cbf1a696291b517. 2023-07-13 06:00:57,309 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:quota,,1689227988210.c8571aee40cfd11e8cbf1a696291b517. after waiting 0 ms 2023-07-13 06:00:57,309 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:quota,,1689227988210.c8571aee40cfd11e8cbf1a696291b517. 2023-07-13 06:00:57,309 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2745): Flushing c8571aee40cfd11e8cbf1a696291b517 2/2 column families, dataSize=21.22 KB heapSize=43.17 KB 2023-07-13 06:00:57,380 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35187] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.regionserver.RegionServerStoppedException: Server jenkins-hbase20.apache.org,35187,1689227983701 stopping at org.apache.hadoop.hbase.regionserver.RSRpcServices.checkOpen(RSRpcServices.java:1527) at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3494) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44996) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-13 06:00:57,380 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35187] ipc.CallRunner(144): callId: 1 service: ClientService methodName: Scan size: 131 connection: 148.251.75.209:52550 deadline: 1689228117270, exception=org.apache.hadoop.hbase.regionserver.RegionServerStoppedException: Server jenkins-hbase20.apache.org,35187,1689227983701 stopping 2023-07-13 06:00:57,401 DEBUG [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(1504): Waiting on 1588230740, b06e3484b8c05bbdc046fc6b5386dddc, c8571aee40cfd11e8cbf1a696291b517, e16a1a1ea32dc6c137274851c98ece04 2023-07-13 06:00:57,584 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=35187] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.regionserver.RegionServerStoppedException: Server jenkins-hbase20.apache.org,35187,1689227983701 stopping at org.apache.hadoop.hbase.regionserver.RSRpcServices.checkOpen(RSRpcServices.java:1527) at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3494) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44996) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-13 06:00:57,584 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=35187] ipc.CallRunner(144): callId: 2 service: ClientService methodName: Scan size: 131 connection: 148.251.75.209:52550 deadline: 1689228117269, exception=org.apache.hadoop.hbase.regionserver.RegionServerStoppedException: Server jenkins-hbase20.apache.org,35187,1689227983701 stopping 2023-07-13 06:00:57,601 DEBUG [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(1504): Waiting on 1588230740, b06e3484b8c05bbdc046fc6b5386dddc, c8571aee40cfd11e8cbf1a696291b517, e16a1a1ea32dc6c137274851c98ece04 2023-07-13 06:00:57,650 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=848 B at sequenceid=57 (bloomFilter=false), to=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/meta/1588230740/.tmp/table/e842d07aa4b94dcc9d5526aa6344e011 2023-07-13 06:00:57,656 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/meta/1588230740/.tmp/info/7952706754f64aeea82db97ba19a3639 as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/meta/1588230740/info/7952706754f64aeea82db97ba19a3639 2023-07-13 06:00:57,665 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/meta/1588230740/info/7952706754f64aeea82db97ba19a3639, entries=160, sequenceid=57, filesize=26.5 K 2023-07-13 06:00:57,666 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/meta/1588230740/.tmp/table/e842d07aa4b94dcc9d5526aa6344e011 as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/meta/1588230740/table/e842d07aa4b94dcc9d5526aa6344e011 2023-07-13 06:00:57,672 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/meta/1588230740/table/e842d07aa4b94dcc9d5526aa6344e011, entries=14, sequenceid=57, filesize=5.4 K 2023-07-13 06:00:57,673 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~22.23 KB/22760, heapSize ~35.48 KB/36328, currentSize=0 B/0 for 1588230740 in 478ms, sequenceid=57, compaction requested=false 2023-07-13 06:00:57,679 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/meta/1588230740/recovered.edits/60.seqid, newMaxSeqId=60, maxSeqId=1 2023-07-13 06:00:57,680 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-13 06:00:57,681 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-13 06:00:57,681 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-13 06:00:57,681 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-13 06:00:57,717 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=328 B at sequenceid=240 (bloomFilter=true), to=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/quota/c8571aee40cfd11e8cbf1a696291b517/.tmp/q/51ec56110fdf4b128b6da281cd5a03e9 2023-07-13 06:00:57,801 DEBUG [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(1504): Waiting on b06e3484b8c05bbdc046fc6b5386dddc, c8571aee40cfd11e8cbf1a696291b517, e16a1a1ea32dc6c137274851c98ece04 2023-07-13 06:00:57,890 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35187] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.regionserver.RegionServerStoppedException: Server jenkins-hbase20.apache.org,35187,1689227983701 stopping at org.apache.hadoop.hbase.regionserver.RSRpcServices.checkOpen(RSRpcServices.java:1527) at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:3494) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44996) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-13 06:00:57,890 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=35187] ipc.CallRunner(144): callId: 3 service: ClientService methodName: Scan size: 131 connection: 148.251.75.209:52550 deadline: 1689228117270, exception=org.apache.hadoop.hbase.regionserver.RegionServerStoppedException: Server jenkins-hbase20.apache.org,35187,1689227983701 stopping 2023-07-13 06:00:58,002 DEBUG [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(1504): Waiting on b06e3484b8c05bbdc046fc6b5386dddc, c8571aee40cfd11e8cbf1a696291b517, e16a1a1ea32dc6c137274851c98ece04 2023-07-13 06:00:58,132 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=20.90 KB at sequenceid=240 (bloomFilter=true), to=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/quota/c8571aee40cfd11e8cbf1a696291b517/.tmp/u/e05c07ba3667419da42ac1bdb018282a 2023-07-13 06:00:58,143 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/quota/c8571aee40cfd11e8cbf1a696291b517/.tmp/q/51ec56110fdf4b128b6da281cd5a03e9 as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/quota/c8571aee40cfd11e8cbf1a696291b517/q/51ec56110fdf4b128b6da281cd5a03e9 2023-07-13 06:00:58,151 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/quota/c8571aee40cfd11e8cbf1a696291b517/q/51ec56110fdf4b128b6da281cd5a03e9, entries=5, sequenceid=240, filesize=5.1 K 2023-07-13 06:00:58,152 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/quota/c8571aee40cfd11e8cbf1a696291b517/.tmp/u/e05c07ba3667419da42ac1bdb018282a as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/quota/c8571aee40cfd11e8cbf1a696291b517/u/e05c07ba3667419da42ac1bdb018282a 2023-07-13 06:00:58,157 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/quota/c8571aee40cfd11e8cbf1a696291b517/u/e05c07ba3667419da42ac1bdb018282a, entries=11, sequenceid=240, filesize=5.7 K 2023-07-13 06:00:58,158 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~21.22 KB/21727, heapSize ~43.14 KB/44176, currentSize=0 B/0 for c8571aee40cfd11e8cbf1a696291b517 in 849ms, sequenceid=240, compaction requested=false 2023-07-13 06:00:58,165 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/hbase/quota/c8571aee40cfd11e8cbf1a696291b517/recovered.edits/243.seqid, newMaxSeqId=243, maxSeqId=1 2023-07-13 06:00:58,166 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed hbase:quota,,1689227988210.c8571aee40cfd11e8cbf1a696291b517. 2023-07-13 06:00:58,166 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for c8571aee40cfd11e8cbf1a696291b517: 2023-07-13 06:00:58,167 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed hbase:quota,,1689227988210.c8571aee40cfd11e8cbf1a696291b517. 2023-07-13 06:00:58,167 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing b06e3484b8c05bbdc046fc6b5386dddc, disabling compactions & flushes 2023-07-13 06:00:58,167 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc. 2023-07-13 06:00:58,167 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc. 2023-07-13 06:00:58,167 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc. after waiting 0 ms 2023-07-13 06:00:58,167 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc. 2023-07-13 06:00:58,171 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/b06e3484b8c05bbdc046fc6b5386dddc/recovered.edits/175.seqid, newMaxSeqId=175, maxSeqId=1 2023-07-13 06:00:58,172 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc. 2023-07-13 06:00:58,172 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for b06e3484b8c05bbdc046fc6b5386dddc: 2023-07-13 06:00:58,172 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed testTablesWithSnapshots4,0,1689228022139.b06e3484b8c05bbdc046fc6b5386dddc. 2023-07-13 06:00:58,173 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing e16a1a1ea32dc6c137274851c98ece04, disabling compactions & flushes 2023-07-13 06:00:58,173 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04. 2023-07-13 06:00:58,173 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04. 2023-07-13 06:00:58,173 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04. after waiting 0 ms 2023-07-13 06:00:58,173 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04. 2023-07-13 06:00:58,177 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/data/default/testTablesWithSnapshots4/e16a1a1ea32dc6c137274851c98ece04/recovered.edits/175.seqid, newMaxSeqId=175, maxSeqId=1 2023-07-13 06:00:58,179 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04. 2023-07-13 06:00:58,179 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for e16a1a1ea32dc6c137274851c98ece04: 2023-07-13 06:00:58,179 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed testTablesWithSnapshots4,5,1689228022139.e16a1a1ea32dc6c137274851c98ece04. 2023-07-13 06:00:58,202 INFO [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(1170): stopping server jenkins-hbase20.apache.org,35187,1689227983701; all regions closed. 2023-07-13 06:00:58,202 DEBUG [RS:0;jenkins-hbase20:35187] quotas.QuotaCache(100): Stopping QuotaRefresherChore chore. 2023-07-13 06:00:58,212 DEBUG [RS:0;jenkins-hbase20:35187] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/oldWALs 2023-07-13 06:00:58,212 INFO [RS:0;jenkins-hbase20:35187] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C35187%2C1689227983701.meta:.meta(num 1689227986385) 2023-07-13 06:00:58,218 DEBUG [RS:0;jenkins-hbase20:35187] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/oldWALs 2023-07-13 06:00:58,218 INFO [RS:0;jenkins-hbase20:35187] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C35187%2C1689227983701:(num 1689227985412) 2023-07-13 06:00:58,218 DEBUG [RS:0;jenkins-hbase20:35187] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:58,218 INFO [RS:0;jenkins-hbase20:35187] regionserver.LeaseManager(133): Closed leases 2023-07-13 06:00:58,219 INFO [RS:0;jenkins-hbase20:35187] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase20:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=5000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-13 06:00:58,219 INFO [regionserver/jenkins-hbase20:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-13 06:00:58,220 INFO [RS:0;jenkins-hbase20:35187] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:35187 2023-07-13 06:00:58,230 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,35187,1689227983701 2023-07-13 06:00:58,230 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-13 06:00:58,230 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-13 06:00:58,231 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase20.apache.org,35187,1689227983701] 2023-07-13 06:00:58,231 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase20.apache.org,35187,1689227983701; numProcessing=1 2023-07-13 06:00:58,232 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase20.apache.org,35187,1689227983701 already deleted, retry=false 2023-07-13 06:00:58,232 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase20.apache.org,35187,1689227983701 expired; onlineServers=0 2023-07-13 06:00:58,232 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase20.apache.org,34737,1689227982756' ***** 2023-07-13 06:00:58,232 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-13 06:00:58,233 DEBUG [M:0;jenkins-hbase20:34737] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6a9977c1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-07-13 06:00:58,233 INFO [M:0;jenkins-hbase20:34737] regionserver.HRegionServer(1144): stopping server jenkins-hbase20.apache.org,34737,1689227982756 2023-07-13 06:00:58,233 INFO [M:0;jenkins-hbase20:34737] regionserver.HRegionServer(1170): stopping server jenkins-hbase20.apache.org,34737,1689227982756; all regions closed. 2023-07-13 06:00:58,233 DEBUG [M:0;jenkins-hbase20:34737] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:58,233 DEBUG [M:0;jenkins-hbase20:34737] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-13 06:00:58,233 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-13 06:00:58,233 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.large.0-1689227985085] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.large.0-1689227985085,5,FailOnTimeoutGroup] 2023-07-13 06:00:58,233 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.small.0-1689227985085] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.small.0-1689227985085,5,FailOnTimeoutGroup] 2023-07-13 06:00:58,233 DEBUG [M:0;jenkins-hbase20:34737] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-13 06:00:58,233 WARN [master/jenkins-hbase20:0.Chore.1] quotas.QuotaRetriever$Iter(131): java.io.InterruptedIOException at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:220) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:53) at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithoutRetries(RpcRetryingCallerImpl.java:187) at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:266) at org.apache.hadoop.hbase.client.ClientScanner.loadCache(ClientScanner.java:435) at org.apache.hadoop.hbase.client.ClientScanner.nextWithSyncCache(ClientScanner.java:309) at org.apache.hadoop.hbase.client.ClientScanner.next(ClientScanner.java:594) at org.apache.hadoop.hbase.client.ConnectionImplementation.locateRegionInMeta(ConnectionImplementation.java:989) at org.apache.hadoop.hbase.client.ConnectionImplementation.locateRegion(ConnectionImplementation.java:866) at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:319) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:157) at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:53) at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithoutRetries(RpcRetryingCallerImpl.java:187) at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:266) at org.apache.hadoop.hbase.client.ClientScanner.loadCache(ClientScanner.java:435) at org.apache.hadoop.hbase.client.ClientScanner.nextWithSyncCache(ClientScanner.java:309) at org.apache.hadoop.hbase.client.ClientScanner.next(ClientScanner.java:594) at org.apache.hadoop.hbase.quotas.QuotaRetriever.next(QuotaRetriever.java:103) at org.apache.hadoop.hbase.quotas.QuotaRetriever$Iter.(QuotaRetriever.java:129) at org.apache.hadoop.hbase.quotas.QuotaRetriever.iterator(QuotaRetriever.java:121) at org.apache.hadoop.hbase.quotas.SnapshotQuotaObserverChore.getSnapshotsToComputeSize(SnapshotQuotaObserverChore.java:171) at org.apache.hadoop.hbase.quotas.SnapshotQuotaObserverChore._chore(SnapshotQuotaObserverChore.java:106) at org.apache.hadoop.hbase.quotas.SnapshotQuotaObserverChore.chore(SnapshotQuotaObserverChore.java:93) at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:107) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-07-13 06:00:58,233 INFO [M:0;jenkins-hbase20:34737] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-13 06:00:58,233 INFO [M:0;jenkins-hbase20:34737] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-13 06:00:58,233 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-07-13 06:00:58,233 WARN [master/jenkins-hbase20:0.Chore.1] quotas.SnapshotQuotaObserverChore(98): Failed to compute the size of snapshots, will retry org.apache.hadoop.hbase.exceptions.MasterStoppedException at org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2843) at org.apache.hadoop.hbase.master.MasterRpcServices.getCompletedSnapshots(MasterRpcServices.java:1002) at org.apache.hadoop.hbase.client.ShortCircuitMasterConnection.getCompletedSnapshots(ShortCircuitMasterConnection.java:496) at org.apache.hadoop.hbase.client.HBaseAdmin$70.rpcCall(HBaseAdmin.java:2793) at org.apache.hadoop.hbase.client.HBaseAdmin$70.rpcCall(HBaseAdmin.java:2789) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:99) at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:104) at org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:2954) at org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:2946) at org.apache.hadoop.hbase.client.HBaseAdmin.listSnapshots(HBaseAdmin.java:2788) at org.apache.hadoop.hbase.quotas.SnapshotQuotaObserverChore.getSnapshotsFromTables(SnapshotQuotaObserverChore.java:199) at org.apache.hadoop.hbase.quotas.SnapshotQuotaObserverChore.getSnapshotsToComputeSize(SnapshotQuotaObserverChore.java:188) at org.apache.hadoop.hbase.quotas.SnapshotQuotaObserverChore._chore(SnapshotQuotaObserverChore.java:106) at org.apache.hadoop.hbase.quotas.SnapshotQuotaObserverChore.chore(SnapshotQuotaObserverChore.java:93) at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:107) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-07-13 06:00:58,233 INFO [M:0;jenkins-hbase20:34737] hbase.ChoreService(369): Chore service for: master/jenkins-hbase20:0 had [ScheduledChore name=QuotaObserverChore, period=1000, unit=MILLISECONDS] on shutdown 2023-07-13 06:00:58,233 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-13 06:00:58,234 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-13 06:00:58,234 DEBUG [M:0;jenkins-hbase20:34737] master.HMaster(1512): Stopping service threads 2023-07-13 06:00:58,234 INFO [M:0;jenkins-hbase20:34737] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-13 06:00:58,235 ERROR [M:0;jenkins-hbase20:34737] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-4,5,PEWorkerGroup] 2023-07-13 06:00:58,235 INFO [M:0;jenkins-hbase20:34737] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-13 06:00:58,235 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-13 06:00:58,235 DEBUG [M:0;jenkins-hbase20:34737] zookeeper.ZKUtil(398): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-07-13 06:00:58,236 WARN [M:0;jenkins-hbase20:34737] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-13 06:00:58,236 INFO [M:0;jenkins-hbase20:34737] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-13 06:00:58,236 INFO [M:0;jenkins-hbase20:34737] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-13 06:00:58,236 DEBUG [M:0;jenkins-hbase20:34737] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-13 06:00:58,236 INFO [M:0;jenkins-hbase20:34737] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-13 06:00:58,236 DEBUG [M:0;jenkins-hbase20:34737] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-13 06:00:58,236 DEBUG [M:0;jenkins-hbase20:34737] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-13 06:00:58,237 DEBUG [M:0;jenkins-hbase20:34737] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-13 06:00:58,237 INFO [M:0;jenkins-hbase20:34737] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=185.51 KB heapSize=221.68 KB 2023-07-13 06:00:58,332 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-13 06:00:58,332 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): regionserver:35187-0x10084e9f1160001, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-13 06:00:58,332 INFO [RS:0;jenkins-hbase20:35187] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase20.apache.org,35187,1689227983701; zookeeper connection closed. 2023-07-13 06:00:58,333 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@36ceb831] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@36ceb831 2023-07-13 06:00:58,333 INFO [Listener at localhost.localdomain/45717] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-07-13 06:00:58,646 INFO [M:0;jenkins-hbase20:34737] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=185.51 KB at sequenceid=420 (bloomFilter=true), to=hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/0123df74cbd345159cad79701163fd88 2023-07-13 06:00:58,650 INFO [M:0;jenkins-hbase20:34737] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0123df74cbd345159cad79701163fd88 2023-07-13 06:00:58,651 DEBUG [M:0;jenkins-hbase20:34737] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/0123df74cbd345159cad79701163fd88 as hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/0123df74cbd345159cad79701163fd88 2023-07-13 06:00:58,656 INFO [M:0;jenkins-hbase20:34737] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0123df74cbd345159cad79701163fd88 2023-07-13 06:00:58,656 INFO [M:0;jenkins-hbase20:34737] regionserver.HStore(1080): Added hdfs://localhost.localdomain:36315/user/jenkins/test-data/2e04fd92-b0ce-5087-31bd-24c174c34729/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/0123df74cbd345159cad79701163fd88, entries=53, sequenceid=420, filesize=13.6 K 2023-07-13 06:00:58,657 INFO [M:0;jenkins-hbase20:34737] regionserver.HRegion(2948): Finished flush of dataSize ~185.51 KB/189959, heapSize ~221.66 KB/226984, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 420ms, sequenceid=420, compaction requested=false 2023-07-13 06:00:58,659 INFO [M:0;jenkins-hbase20:34737] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-13 06:00:58,659 DEBUG [M:0;jenkins-hbase20:34737] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-13 06:00:58,662 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-13 06:00:58,662 INFO [M:0;jenkins-hbase20:34737] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-13 06:00:58,663 INFO [M:0;jenkins-hbase20:34737] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:34737 2023-07-13 06:00:58,664 DEBUG [M:0;jenkins-hbase20:34737] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase20.apache.org,34737,1689227982756 already deleted, retry=false 2023-07-13 06:00:58,766 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-13 06:00:58,766 INFO [M:0;jenkins-hbase20:34737] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase20.apache.org,34737,1689227982756; zookeeper connection closed. 2023-07-13 06:00:58,766 DEBUG [Listener at localhost.localdomain/45717-EventThread] zookeeper.ZKWatcher(600): master:34737-0x10084e9f1160000, quorum=127.0.0.1:55963, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-13 06:00:58,770 WARN [Listener at localhost.localdomain/45717] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-13 06:00:58,775 INFO [Listener at localhost.localdomain/45717] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-13 06:00:58,888 WARN [BP-62436662-148.251.75.209-1689227979967 heartbeating to localhost.localdomain/127.0.0.1:36315] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-13 06:00:58,888 WARN [BP-62436662-148.251.75.209-1689227979967 heartbeating to localhost.localdomain/127.0.0.1:36315] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-62436662-148.251.75.209-1689227979967 (Datanode Uuid 7f8df910-a3c0-47a0-8fe6-894f5b42ccb2) service to localhost.localdomain/127.0.0.1:36315 2023-07-13 06:00:58,890 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/74b62a1d-1af8-4c02-70de-8b0007eeebf7/cluster_3a5fcc1f-2335-da3f-c526-d77046317035/dfs/data/data1/current/BP-62436662-148.251.75.209-1689227979967] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-13 06:00:58,891 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/74b62a1d-1af8-4c02-70de-8b0007eeebf7/cluster_3a5fcc1f-2335-da3f-c526-d77046317035/dfs/data/data2/current/BP-62436662-148.251.75.209-1689227979967] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-13 06:00:59,000 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5c8010cf to 127.0.0.1:55963 2023-07-13 06:00:59,000 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:59,000 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x285e2c13 to 127.0.0.1:55963 2023-07-13 06:00:59,000 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:59,000 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x46c45555 to 127.0.0.1:55963 2023-07-13 06:00:59,000 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:59,001 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x30bca37c to 127.0.0.1:55963 2023-07-13 06:00:59,001 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:59,001 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x20d3db4b to 127.0.0.1:55963 2023-07-13 06:00:59,001 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:59,001 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x57445309 to 127.0.0.1:55963 2023-07-13 06:00:59,001 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:59,001 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2495f85e to 127.0.0.1:55963 2023-07-13 06:00:59,001 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-13 06:00:59,029 INFO [Listener at localhost.localdomain/45717] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:0 2023-07-13 06:00:59,150 INFO [Listener at localhost.localdomain/45717] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-07-13 06:00:59,198 INFO [Listener at localhost.localdomain/45717] hbase.HBaseTestingUtility(1293): Minicluster is down