2023-12-10 06:55:48,127 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.quotas.TestClusterScopeQuotaThrottle timeout: 13 mins 2023-12-10 06:55:48,355 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/32556c7f-b21a-6d60-9914-e7d129008d6d 2023-12-10 06:55:48,370 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=2, rsPorts=, rsClass=null, numDataNodes=2, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-12-10 06:55:48,370 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/32556c7f-b21a-6d60-9914-e7d129008d6d/cluster_4172d4ef-86cd-605e-0704-d776bf6f94c3, deleteOnExit=true 2023-12-10 06:55:48,370 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-12-10 06:55:48,371 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/32556c7f-b21a-6d60-9914-e7d129008d6d/test.cache.data in system properties and HBase conf 2023-12-10 06:55:48,371 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/32556c7f-b21a-6d60-9914-e7d129008d6d/hadoop.tmp.dir in system properties and HBase conf 2023-12-10 06:55:48,372 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/32556c7f-b21a-6d60-9914-e7d129008d6d/hadoop.log.dir in system properties and HBase conf 2023-12-10 06:55:48,372 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/32556c7f-b21a-6d60-9914-e7d129008d6d/mapreduce.cluster.local.dir in system properties and HBase conf 2023-12-10 06:55:48,373 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/32556c7f-b21a-6d60-9914-e7d129008d6d/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-12-10 06:55:48,373 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-12-10 06:55:48,480 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-12-10 06:55:48,858 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-12-10 06:55:48,863 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/32556c7f-b21a-6d60-9914-e7d129008d6d/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-12-10 06:55:48,863 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/32556c7f-b21a-6d60-9914-e7d129008d6d/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-12-10 06:55:48,863 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/32556c7f-b21a-6d60-9914-e7d129008d6d/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-12-10 06:55:48,863 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/32556c7f-b21a-6d60-9914-e7d129008d6d/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-12-10 06:55:48,864 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/32556c7f-b21a-6d60-9914-e7d129008d6d/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-12-10 06:55:48,864 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/32556c7f-b21a-6d60-9914-e7d129008d6d/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-12-10 06:55:48,864 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/32556c7f-b21a-6d60-9914-e7d129008d6d/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-12-10 06:55:48,865 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/32556c7f-b21a-6d60-9914-e7d129008d6d/dfs.journalnode.edits.dir in system properties and HBase conf 2023-12-10 06:55:48,865 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/32556c7f-b21a-6d60-9914-e7d129008d6d/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-12-10 06:55:48,866 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/32556c7f-b21a-6d60-9914-e7d129008d6d/nfs.dump.dir in system properties and HBase conf 2023-12-10 06:55:48,866 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/32556c7f-b21a-6d60-9914-e7d129008d6d/java.io.tmpdir in system properties and HBase conf 2023-12-10 06:55:48,867 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/32556c7f-b21a-6d60-9914-e7d129008d6d/dfs.journalnode.edits.dir in system properties and HBase conf 2023-12-10 06:55:48,867 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/32556c7f-b21a-6d60-9914-e7d129008d6d/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-12-10 06:55:48,867 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/32556c7f-b21a-6d60-9914-e7d129008d6d/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-12-10 06:55:49,427 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-12-10 06:55:49,432 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-12-10 06:55:49,717 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-12-10 06:55:49,884 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-12-10 06:55:49,904 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-12-10 06:55:49,946 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-12-10 06:55:49,977 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/32556c7f-b21a-6d60-9914-e7d129008d6d/java.io.tmpdir/Jetty_localhost_40185_hdfs____.ik86zo/webapp 2023-12-10 06:55:50,124 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40185 2023-12-10 06:55:50,135 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-12-10 06:55:50,135 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-12-10 06:55:50,606 WARN [Listener at localhost/39271] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-12-10 06:55:50,668 WARN [Listener at localhost/39271] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-12-10 06:55:50,685 WARN [Listener at localhost/39271] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-12-10 06:55:50,692 INFO [Listener at localhost/39271] log.Slf4jLog(67): jetty-6.1.26 2023-12-10 06:55:50,697 INFO [Listener at localhost/39271] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/32556c7f-b21a-6d60-9914-e7d129008d6d/java.io.tmpdir/Jetty_localhost_45909_datanode____9kv1vs/webapp 2023-12-10 06:55:50,794 INFO [Listener at localhost/39271] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45909 2023-12-10 06:55:51,088 WARN [Listener at localhost/37161] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-12-10 06:55:51,102 WARN [Listener at localhost/37161] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-12-10 06:55:51,108 WARN [Listener at localhost/37161] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-12-10 06:55:51,110 INFO [Listener at localhost/37161] log.Slf4jLog(67): jetty-6.1.26 2023-12-10 06:55:51,117 INFO [Listener at localhost/37161] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/32556c7f-b21a-6d60-9914-e7d129008d6d/java.io.tmpdir/Jetty_localhost_32821_datanode____9oxtpb/webapp 2023-12-10 06:55:51,236 INFO [Listener at localhost/37161] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:32821 2023-12-10 06:55:51,248 WARN [Listener at localhost/38369] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-12-10 06:55:51,534 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x3e3c03ad657a1c0e: Processing first storage report for DS-36bf9369-3517-4182-8ab9-3e7ad5cf4619 from datanode f6ef735b-668b-4f1c-b700-62801aa08b11 2023-12-10 06:55:51,536 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x3e3c03ad657a1c0e: from storage DS-36bf9369-3517-4182-8ab9-3e7ad5cf4619 node DatanodeRegistration(127.0.0.1:37723, datanodeUuid=f6ef735b-668b-4f1c-b700-62801aa08b11, infoPort=33025, infoSecurePort=0, ipcPort=38369, storageInfo=lv=-57;cid=testClusterID;nsid=576605667;c=1702191349506), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-12-10 06:55:51,536 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x6cdc57c96c533ee3: Processing first storage report for DS-966649dc-c536-4785-ba4f-0358770eeb5a from datanode 2ffdebb3-2bf6-4e4c-8e40-1e05648d3392 2023-12-10 06:55:51,536 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x6cdc57c96c533ee3: from storage DS-966649dc-c536-4785-ba4f-0358770eeb5a node DatanodeRegistration(127.0.0.1:33127, datanodeUuid=2ffdebb3-2bf6-4e4c-8e40-1e05648d3392, infoPort=45301, infoSecurePort=0, ipcPort=37161, storageInfo=lv=-57;cid=testClusterID;nsid=576605667;c=1702191349506), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-12-10 06:55:51,536 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x3e3c03ad657a1c0e: Processing first storage report for DS-dac5bbb2-e515-4b8a-a781-86f1716ba3de from datanode f6ef735b-668b-4f1c-b700-62801aa08b11 2023-12-10 06:55:51,536 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x3e3c03ad657a1c0e: from storage DS-dac5bbb2-e515-4b8a-a781-86f1716ba3de node DatanodeRegistration(127.0.0.1:37723, datanodeUuid=f6ef735b-668b-4f1c-b700-62801aa08b11, infoPort=33025, infoSecurePort=0, ipcPort=38369, storageInfo=lv=-57;cid=testClusterID;nsid=576605667;c=1702191349506), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-12-10 06:55:51,536 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x6cdc57c96c533ee3: Processing first storage report for DS-65a716af-3595-49ff-a9fe-e618dff33bb0 from datanode 2ffdebb3-2bf6-4e4c-8e40-1e05648d3392 2023-12-10 06:55:51,536 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x6cdc57c96c533ee3: from storage DS-65a716af-3595-49ff-a9fe-e618dff33bb0 node DatanodeRegistration(127.0.0.1:33127, datanodeUuid=2ffdebb3-2bf6-4e4c-8e40-1e05648d3392, infoPort=45301, infoSecurePort=0, ipcPort=37161, storageInfo=lv=-57;cid=testClusterID;nsid=576605667;c=1702191349506), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-12-10 06:55:51,625 DEBUG [Listener at localhost/38369] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/32556c7f-b21a-6d60-9914-e7d129008d6d 2023-12-10 06:55:51,733 INFO [Listener at localhost/38369] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/32556c7f-b21a-6d60-9914-e7d129008d6d/cluster_4172d4ef-86cd-605e-0704-d776bf6f94c3/zookeeper_0, clientPort=51670, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/32556c7f-b21a-6d60-9914-e7d129008d6d/cluster_4172d4ef-86cd-605e-0704-d776bf6f94c3/zookeeper_0/version-2, dataDirSize=457 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/32556c7f-b21a-6d60-9914-e7d129008d6d/cluster_4172d4ef-86cd-605e-0704-d776bf6f94c3/zookeeper_0/version-2, dataLogSize=457 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, clientPortListenBacklog=-1, serverId=0 2023-12-10 06:55:51,749 INFO [Listener at localhost/38369] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=51670 2023-12-10 06:55:51,757 INFO [Listener at localhost/38369] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-12-10 06:55:51,760 INFO [Listener at localhost/38369] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-12-10 06:55:52,430 INFO [Listener at localhost/38369] util.FSUtils(471): Created version file at hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2 with version=8 2023-12-10 06:55:52,430 INFO [Listener at localhost/38369] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/hbase-staging 2023-12-10 06:55:52,727 INFO [Listener at localhost/38369] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-12-10 06:55:53,176 INFO [Listener at localhost/38369] client.ConnectionUtils(126): master/jenkins-hbase4:0 server-side Connection retries=45 2023-12-10 06:55:53,207 INFO [Listener at localhost/38369] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-12-10 06:55:53,208 INFO [Listener at localhost/38369] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-12-10 06:55:53,208 INFO [Listener at localhost/38369] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-12-10 06:55:53,208 INFO [Listener at localhost/38369] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-12-10 06:55:53,208 INFO [Listener at localhost/38369] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-12-10 06:55:53,348 INFO [Listener at localhost/38369] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-12-10 06:55:53,418 DEBUG [Listener at localhost/38369] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-12-10 06:55:53,508 INFO [Listener at localhost/38369] ipc.NettyRpcServer(120): Bind to /172.31.14.131:40533 2023-12-10 06:55:53,517 INFO [Listener at localhost/38369] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-12-10 06:55:53,520 INFO [Listener at localhost/38369] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-12-10 06:55:53,542 INFO [Listener at localhost/38369] zookeeper.RecoverableZooKeeper(93): Process identifier=master:40533 connecting to ZooKeeper ensemble=127.0.0.1:51670 2023-12-10 06:55:53,583 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): master:405330x0, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-12-10 06:55:53,584 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:40533-0x1007092abbd0000 connected 2023-12-10 06:55:53,619 DEBUG [Listener at localhost/38369] zookeeper.ZKUtil(165): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-12-10 06:55:53,620 DEBUG [Listener at localhost/38369] zookeeper.ZKUtil(165): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-12-10 06:55:53,623 DEBUG [Listener at localhost/38369] zookeeper.ZKUtil(165): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-12-10 06:55:53,631 DEBUG [Listener at localhost/38369] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=40533 2023-12-10 06:55:53,631 DEBUG [Listener at localhost/38369] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=40533 2023-12-10 06:55:53,631 DEBUG [Listener at localhost/38369] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=40533 2023-12-10 06:55:53,632 DEBUG [Listener at localhost/38369] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=40533 2023-12-10 06:55:53,632 DEBUG [Listener at localhost/38369] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=40533 2023-12-10 06:55:53,638 INFO [Listener at localhost/38369] master.HMaster(445): hbase.rootdir=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2, hbase.cluster.distributed=false 2023-12-10 06:55:53,703 INFO [Listener at localhost/38369] client.ConnectionUtils(126): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-12-10 06:55:53,703 INFO [Listener at localhost/38369] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-12-10 06:55:53,704 INFO [Listener at localhost/38369] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-12-10 06:55:53,704 INFO [Listener at localhost/38369] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-12-10 06:55:53,704 INFO [Listener at localhost/38369] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-12-10 06:55:53,704 INFO [Listener at localhost/38369] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-12-10 06:55:53,709 INFO [Listener at localhost/38369] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-12-10 06:55:53,711 INFO [Listener at localhost/38369] ipc.NettyRpcServer(120): Bind to /172.31.14.131:37997 2023-12-10 06:55:53,713 INFO [Listener at localhost/38369] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-12-10 06:55:53,719 DEBUG [Listener at localhost/38369] mob.MobFileCache(121): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-12-10 06:55:53,720 INFO [Listener at localhost/38369] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-12-10 06:55:53,722 INFO [Listener at localhost/38369] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-12-10 06:55:53,724 INFO [Listener at localhost/38369] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:37997 connecting to ZooKeeper ensemble=127.0.0.1:51670 2023-12-10 06:55:53,728 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): regionserver:379970x0, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-12-10 06:55:53,729 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:37997-0x1007092abbd0001 connected 2023-12-10 06:55:53,729 DEBUG [Listener at localhost/38369] zookeeper.ZKUtil(165): regionserver:37997-0x1007092abbd0001, quorum=127.0.0.1:51670, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-12-10 06:55:53,730 DEBUG [Listener at localhost/38369] zookeeper.ZKUtil(165): regionserver:37997-0x1007092abbd0001, quorum=127.0.0.1:51670, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-12-10 06:55:53,731 DEBUG [Listener at localhost/38369] zookeeper.ZKUtil(165): regionserver:37997-0x1007092abbd0001, quorum=127.0.0.1:51670, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-12-10 06:55:53,731 DEBUG [Listener at localhost/38369] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37997 2023-12-10 06:55:53,732 DEBUG [Listener at localhost/38369] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37997 2023-12-10 06:55:53,733 DEBUG [Listener at localhost/38369] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37997 2023-12-10 06:55:53,733 DEBUG [Listener at localhost/38369] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37997 2023-12-10 06:55:53,733 DEBUG [Listener at localhost/38369] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37997 2023-12-10 06:55:53,745 INFO [Listener at localhost/38369] client.ConnectionUtils(126): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-12-10 06:55:53,746 INFO [Listener at localhost/38369] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-12-10 06:55:53,746 INFO [Listener at localhost/38369] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-12-10 06:55:53,746 INFO [Listener at localhost/38369] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-12-10 06:55:53,746 INFO [Listener at localhost/38369] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-12-10 06:55:53,747 INFO [Listener at localhost/38369] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-12-10 06:55:53,747 INFO [Listener at localhost/38369] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-12-10 06:55:53,749 INFO [Listener at localhost/38369] ipc.NettyRpcServer(120): Bind to /172.31.14.131:42073 2023-12-10 06:55:53,749 INFO [Listener at localhost/38369] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-12-10 06:55:53,750 DEBUG [Listener at localhost/38369] mob.MobFileCache(121): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-12-10 06:55:53,751 INFO [Listener at localhost/38369] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-12-10 06:55:53,753 INFO [Listener at localhost/38369] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-12-10 06:55:53,755 INFO [Listener at localhost/38369] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:42073 connecting to ZooKeeper ensemble=127.0.0.1:51670 2023-12-10 06:55:53,759 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): regionserver:420730x0, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-12-10 06:55:53,760 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:42073-0x1007092abbd0002 connected 2023-12-10 06:55:53,760 DEBUG [Listener at localhost/38369] zookeeper.ZKUtil(165): regionserver:42073-0x1007092abbd0002, quorum=127.0.0.1:51670, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-12-10 06:55:53,761 DEBUG [Listener at localhost/38369] zookeeper.ZKUtil(165): regionserver:42073-0x1007092abbd0002, quorum=127.0.0.1:51670, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-12-10 06:55:53,761 DEBUG [Listener at localhost/38369] zookeeper.ZKUtil(165): regionserver:42073-0x1007092abbd0002, quorum=127.0.0.1:51670, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-12-10 06:55:53,762 DEBUG [Listener at localhost/38369] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=42073 2023-12-10 06:55:53,762 DEBUG [Listener at localhost/38369] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=42073 2023-12-10 06:55:53,763 DEBUG [Listener at localhost/38369] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=42073 2023-12-10 06:55:53,764 DEBUG [Listener at localhost/38369] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=42073 2023-12-10 06:55:53,764 DEBUG [Listener at localhost/38369] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=42073 2023-12-10 06:55:53,766 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2174): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,40533,1702191352570 2023-12-10 06:55:53,776 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-12-10 06:55:53,778 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(163): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,40533,1702191352570 2023-12-10 06:55:53,796 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): regionserver:42073-0x1007092abbd0002, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-12-10 06:55:53,796 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): regionserver:37997-0x1007092abbd0001, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-12-10 06:55:53,796 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-12-10 06:55:53,797 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-12-10 06:55:53,798 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(163): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-12-10 06:55:53,799 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,40533,1702191352570 from backup master directory 2023-12-10 06:55:53,799 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(163): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-12-10 06:55:53,802 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,40533,1702191352570 2023-12-10 06:55:53,802 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-12-10 06:55:53,804 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-12-10 06:55:53,804 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,40533,1702191352570 2023-12-10 06:55:53,807 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-12-10 06:55:53,808 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-12-10 06:55:53,889 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/hbase.id with ID: 6abf3012-fb1d-406b-a75e-ff6d635accf5 2023-12-10 06:55:53,931 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-12-10 06:55:53,947 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-12-10 06:55:53,996 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x61d15571 to 127.0.0.1:51670 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-12-10 06:55:54,031 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7da32520, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-12-10 06:55:54,054 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-12-10 06:55:54,056 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-12-10 06:55:54,074 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-12-10 06:55:54,074 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-12-10 06:55:54,076 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:856) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2199) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:529) at java.lang.Thread.run(Thread.java:750) 2023-12-10 06:55:54,080 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:856) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2199) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:529) at java.lang.Thread.run(Thread.java:750) 2023-12-10 06:55:54,081 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-12-10 06:55:54,114 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/MasterData/data/master/store-tmp 2023-12-10 06:55:54,145 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-12-10 06:55:54,146 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-12-10 06:55:54,146 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-12-10 06:55:54,146 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-12-10 06:55:54,146 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-12-10 06:55:54,146 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-12-10 06:55:54,146 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-12-10 06:55:54,146 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-12-10 06:55:54,147 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/MasterData/WALs/jenkins-hbase4.apache.org,40533,1702191352570 2023-12-10 06:55:54,166 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C40533%2C1702191352570, suffix=, logDir=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/MasterData/WALs/jenkins-hbase4.apache.org,40533,1702191352570, archiveDir=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/MasterData/oldWALs, maxLogs=10 2023-12-10 06:55:54,223 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37723,DS-36bf9369-3517-4182-8ab9-3e7ad5cf4619,DISK] 2023-12-10 06:55:54,223 DEBUG [RS-EventLoopGroup-4-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33127,DS-966649dc-c536-4785-ba4f-0358770eeb5a,DISK] 2023-12-10 06:55:54,231 DEBUG [RS-EventLoopGroup-4-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:407) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-12-10 06:55:54,293 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/MasterData/WALs/jenkins-hbase4.apache.org,40533,1702191352570/jenkins-hbase4.apache.org%2C40533%2C1702191352570.1702191354176 2023-12-10 06:55:54,294 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37723,DS-36bf9369-3517-4182-8ab9-3e7ad5cf4619,DISK], DatanodeInfoWithStorage[127.0.0.1:33127,DS-966649dc-c536-4785-ba4f-0358770eeb5a,DISK]] 2023-12-10 06:55:54,294 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-12-10 06:55:54,294 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-12-10 06:55:54,297 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-12-10 06:55:54,299 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-12-10 06:55:54,352 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-12-10 06:55:54,360 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-12-10 06:55:54,382 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:10, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-12-10 06:55:54,393 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-12-10 06:55:54,399 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-12-10 06:55:54,401 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-12-10 06:55:54,414 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-12-10 06:55:54,417 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-12-10 06:55:54,418 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75275104, jitterRate=0.12168645858764648}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-12-10 06:55:54,419 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-12-10 06:55:54,419 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-12-10 06:55:54,437 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-12-10 06:55:54,437 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(608): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-12-10 06:55:54,439 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-12-10 06:55:54,441 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(628): Recovered RegionProcedureStore lease in 0 msec 2023-12-10 06:55:54,472 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(642): Loaded RegionProcedureStore in 31 msec 2023-12-10 06:55:54,472 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-12-10 06:55:54,497 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-12-10 06:55:54,501 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-12-10 06:55:54,525 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-12-10 06:55:54,528 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-12-10 06:55:54,529 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(165): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-12-10 06:55:54,534 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-12-10 06:55:54,538 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(165): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-12-10 06:55:54,542 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-12-10 06:55:54,543 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(165): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-12-10 06:55:54,544 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(165): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-12-10 06:55:54,559 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(165): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-12-10 06:55:54,565 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): regionserver:42073-0x1007092abbd0002, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-12-10 06:55:54,565 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-12-10 06:55:54,565 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): regionserver:37997-0x1007092abbd0001, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-12-10 06:55:54,565 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-12-10 06:55:54,566 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(745): Active/primary master=jenkins-hbase4.apache.org,40533,1702191352570, sessionid=0x1007092abbd0000, setting cluster-up flag (Was=false) 2023-12-10 06:55:54,583 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-12-10 06:55:54,589 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-12-10 06:55:54,591 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,40533,1702191352570 2023-12-10 06:55:54,596 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-12-10 06:55:54,605 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-12-10 06:55:54,607 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,40533,1702191352570 2023-12-10 06:55:54,609 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(304): Couldn't delete working snapshot directory: hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.hbase-snapshot/.tmp 2023-12-10 06:55:54,650 INFO [master/jenkins-hbase4:0:becomeActiveMaster] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.quotas.MasterQuotasObserver loaded, priority=536870911. 2023-12-10 06:55:54,667 INFO [RS:0;jenkins-hbase4:37997] regionserver.HRegionServer(954): ClusterId : 6abf3012-fb1d-406b-a75e-ff6d635accf5 2023-12-10 06:55:54,667 INFO [RS:1;jenkins-hbase4:42073] regionserver.HRegionServer(954): ClusterId : 6abf3012-fb1d-406b-a75e-ff6d635accf5 2023-12-10 06:55:54,673 DEBUG [RS:0;jenkins-hbase4:37997] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-12-10 06:55:54,673 DEBUG [RS:1;jenkins-hbase4:42073] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-12-10 06:55:54,679 DEBUG [RS:1;jenkins-hbase4:42073] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-12-10 06:55:54,679 DEBUG [RS:0;jenkins-hbase4:37997] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-12-10 06:55:54,679 DEBUG [RS:1;jenkins-hbase4:42073] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-12-10 06:55:54,679 DEBUG [RS:0;jenkins-hbase4:37997] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-12-10 06:55:54,683 DEBUG [RS:1;jenkins-hbase4:42073] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-12-10 06:55:54,684 DEBUG [RS:0;jenkins-hbase4:37997] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-12-10 06:55:54,691 DEBUG [RS:1;jenkins-hbase4:42073] zookeeper.ReadOnlyZKClient(139): Connect 0x7978c06b to 127.0.0.1:51670 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-12-10 06:55:54,691 DEBUG [RS:0;jenkins-hbase4:37997] zookeeper.ReadOnlyZKClient(139): Connect 0x1e40ef1e to 127.0.0.1:51670 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-12-10 06:55:54,698 DEBUG [RS:1;jenkins-hbase4:42073] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@77bf2134, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-12-10 06:55:54,699 DEBUG [RS:1;jenkins-hbase4:42073] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7380ab1b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-12-10 06:55:54,699 DEBUG [RS:0;jenkins-hbase4:37997] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@108712a6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-12-10 06:55:54,699 DEBUG [RS:0;jenkins-hbase4:37997] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@14b8846e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-12-10 06:55:54,721 DEBUG [RS:1;jenkins-hbase4:42073] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:42073 2023-12-10 06:55:54,722 DEBUG [RS:0;jenkins-hbase4:37997] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:37997 2023-12-10 06:55:54,725 INFO [RS:1;jenkins-hbase4:42073] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-12-10 06:55:54,725 INFO [RS:1;jenkins-hbase4:42073] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-12-10 06:55:54,725 INFO [RS:0;jenkins-hbase4:37997] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-12-10 06:55:54,725 INFO [RS:0;jenkins-hbase4:37997] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-12-10 06:55:54,725 DEBUG [RS:1;jenkins-hbase4:42073] regionserver.HRegionServer(1025): About to register with Master. 2023-12-10 06:55:54,725 DEBUG [RS:0;jenkins-hbase4:37997] regionserver.HRegionServer(1025): About to register with Master. 2023-12-10 06:55:54,728 INFO [RS:1;jenkins-hbase4:42073] regionserver.HRegionServer(2814): reportForDuty to master=jenkins-hbase4.apache.org,40533,1702191352570 with isa=jenkins-hbase4.apache.org/172.31.14.131:42073, startcode=1702191353745 2023-12-10 06:55:54,728 INFO [RS:0;jenkins-hbase4:37997] regionserver.HRegionServer(2814): reportForDuty to master=jenkins-hbase4.apache.org,40533,1702191352570 with isa=jenkins-hbase4.apache.org/172.31.14.131:37997, startcode=1702191353702 2023-12-10 06:55:54,738 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1075): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-12-10 06:55:54,744 DEBUG [RS:0;jenkins-hbase4:37997] ipc.RpcConnection(122): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-12-10 06:55:54,744 DEBUG [RS:1;jenkins-hbase4:42073] ipc.RpcConnection(122): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-12-10 06:55:54,747 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-12-10 06:55:54,748 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-12-10 06:55:54,748 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-12-10 06:55:54,748 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-12-10 06:55:54,748 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-12-10 06:55:54,748 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-12-10 06:55:54,748 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-12-10 06:55:54,748 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-12-10 06:55:54,750 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1702191384750 2023-12-10 06:55:54,753 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-12-10 06:55:54,758 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-12-10 06:55:54,758 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-12-10 06:55:54,763 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-12-10 06:55:54,763 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-12-10 06:55:54,770 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-12-10 06:55:54,771 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-12-10 06:55:54,771 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-12-10 06:55:54,771 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-12-10 06:55:54,775 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-12-10 06:55:54,777 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-12-10 06:55:54,779 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-12-10 06:55:54,779 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-12-10 06:55:54,787 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-12-10 06:55:54,787 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-12-10 06:55:54,790 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1702191354790,5,FailOnTimeoutGroup] 2023-12-10 06:55:54,791 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1702191354790,5,FailOnTimeoutGroup] 2023-12-10 06:55:54,791 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-12-10 06:55:54,791 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1466): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-12-10 06:55:54,794 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-12-10 06:55:54,794 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-12-10 06:55:54,808 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57329, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-12-10 06:55:54,808 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:33643, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-12-10 06:55:54,813 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-12-10 06:55:54,815 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-12-10 06:55:54,815 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2 2023-12-10 06:55:54,821 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=40533] master.ServerManager(388): Registering regionserver=jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 06:55:54,835 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=40533] master.ServerManager(388): Registering regionserver=jenkins-hbase4.apache.org,42073,1702191353745 2023-12-10 06:55:54,841 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-12-10 06:55:54,842 DEBUG [RS:1;jenkins-hbase4:42073] regionserver.HRegionServer(1598): Config from master: hbase.rootdir=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2 2023-12-10 06:55:54,842 DEBUG [RS:0;jenkins-hbase4:37997] regionserver.HRegionServer(1598): Config from master: hbase.rootdir=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2 2023-12-10 06:55:54,843 DEBUG [RS:1;jenkins-hbase4:42073] regionserver.HRegionServer(1598): Config from master: fs.defaultFS=hdfs://localhost:39271 2023-12-10 06:55:54,843 DEBUG [RS:0;jenkins-hbase4:37997] regionserver.HRegionServer(1598): Config from master: fs.defaultFS=hdfs://localhost:39271 2023-12-10 06:55:54,843 DEBUG [RS:1;jenkins-hbase4:42073] regionserver.HRegionServer(1598): Config from master: hbase.master.info.port=-1 2023-12-10 06:55:54,843 DEBUG [RS:0;jenkins-hbase4:37997] regionserver.HRegionServer(1598): Config from master: hbase.master.info.port=-1 2023-12-10 06:55:54,844 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-12-10 06:55:54,847 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740/info 2023-12-10 06:55:54,848 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:10, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-12-10 06:55:54,849 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-12-10 06:55:54,849 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-12-10 06:55:54,852 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-12-10 06:55:54,852 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740/rep_barrier 2023-12-10 06:55:54,853 DEBUG [RS:0;jenkins-hbase4:37997] zookeeper.ZKUtil(163): regionserver:37997-0x1007092abbd0001, quorum=127.0.0.1:51670, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 06:55:54,853 WARN [RS:0;jenkins-hbase4:37997] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-12-10 06:55:54,853 DEBUG [RS:1;jenkins-hbase4:42073] zookeeper.ZKUtil(163): regionserver:42073-0x1007092abbd0002, quorum=127.0.0.1:51670, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42073,1702191353745 2023-12-10 06:55:54,853 INFO [RS:0;jenkins-hbase4:37997] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-12-10 06:55:54,853 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:10, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-12-10 06:55:54,854 DEBUG [RS:0;jenkins-hbase4:37997] regionserver.HRegionServer(1951): logDir=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/WALs/jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 06:55:54,854 WARN [RS:1;jenkins-hbase4:42073] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-12-10 06:55:54,855 INFO [RS:1;jenkins-hbase4:42073] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-12-10 06:55:54,856 DEBUG [RS:1;jenkins-hbase4:42073] regionserver.HRegionServer(1951): logDir=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/WALs/jenkins-hbase4.apache.org,42073,1702191353745 2023-12-10 06:55:54,856 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,37997,1702191353702] 2023-12-10 06:55:54,856 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,42073,1702191353745] 2023-12-10 06:55:54,856 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-12-10 06:55:54,857 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-12-10 06:55:54,862 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740/table 2023-12-10 06:55:54,863 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:10, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-12-10 06:55:54,864 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-12-10 06:55:54,865 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740 2023-12-10 06:55:54,866 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740 2023-12-10 06:55:54,870 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-12-10 06:55:54,871 DEBUG [RS:0;jenkins-hbase4:37997] zookeeper.ZKUtil(163): regionserver:37997-0x1007092abbd0001, quorum=127.0.0.1:51670, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 06:55:54,871 DEBUG [RS:1;jenkins-hbase4:42073] zookeeper.ZKUtil(163): regionserver:42073-0x1007092abbd0002, quorum=127.0.0.1:51670, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 06:55:54,872 DEBUG [RS:0;jenkins-hbase4:37997] zookeeper.ZKUtil(163): regionserver:37997-0x1007092abbd0001, quorum=127.0.0.1:51670, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42073,1702191353745 2023-12-10 06:55:54,872 DEBUG [RS:1;jenkins-hbase4:42073] zookeeper.ZKUtil(163): regionserver:42073-0x1007092abbd0002, quorum=127.0.0.1:51670, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,42073,1702191353745 2023-12-10 06:55:54,872 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-12-10 06:55:54,876 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-12-10 06:55:54,877 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71426941, jitterRate=0.0643443614244461}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-12-10 06:55:54,877 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-12-10 06:55:54,877 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-12-10 06:55:54,877 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-12-10 06:55:54,877 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-12-10 06:55:54,877 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-12-10 06:55:54,877 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-12-10 06:55:54,878 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-12-10 06:55:54,878 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-12-10 06:55:54,883 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-12-10 06:55:54,883 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-12-10 06:55:54,884 DEBUG [RS:1;jenkins-hbase4:42073] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-12-10 06:55:54,884 DEBUG [RS:0;jenkins-hbase4:37997] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-12-10 06:55:54,891 INFO [PEWorker-1] procedure2.ProcedureExecutor(1796): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-12-10 06:55:54,893 INFO [RS:1;jenkins-hbase4:42073] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-12-10 06:55:54,893 INFO [RS:0;jenkins-hbase4:37997] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-12-10 06:55:54,902 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-12-10 06:55:54,905 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-12-10 06:55:54,913 INFO [RS:0;jenkins-hbase4:37997] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-12-10 06:55:54,913 INFO [RS:1;jenkins-hbase4:42073] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-12-10 06:55:54,917 INFO [RS:1;jenkins-hbase4:42073] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-12-10 06:55:54,917 INFO [RS:0;jenkins-hbase4:37997] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-12-10 06:55:54,917 INFO [RS:1;jenkins-hbase4:42073] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-12-10 06:55:54,917 INFO [RS:0;jenkins-hbase4:37997] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-12-10 06:55:54,919 INFO [RS:1;jenkins-hbase4:42073] regionserver.HRegionServer$CompactionChecker(1840): CompactionChecker runs every PT1S 2023-12-10 06:55:54,919 INFO [RS:0;jenkins-hbase4:37997] regionserver.HRegionServer$CompactionChecker(1840): CompactionChecker runs every PT1S 2023-12-10 06:55:54,927 INFO [RS:1;jenkins-hbase4:42073] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-12-10 06:55:54,927 INFO [RS:0;jenkins-hbase4:37997] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-12-10 06:55:54,927 DEBUG [RS:1;jenkins-hbase4:42073] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-12-10 06:55:54,927 DEBUG [RS:0;jenkins-hbase4:37997] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-12-10 06:55:54,927 DEBUG [RS:1;jenkins-hbase4:42073] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-12-10 06:55:54,927 DEBUG [RS:0;jenkins-hbase4:37997] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-12-10 06:55:54,927 DEBUG [RS:1;jenkins-hbase4:42073] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-12-10 06:55:54,927 DEBUG [RS:0;jenkins-hbase4:37997] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-12-10 06:55:54,927 DEBUG [RS:1;jenkins-hbase4:42073] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-12-10 06:55:54,928 DEBUG [RS:0;jenkins-hbase4:37997] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-12-10 06:55:54,928 DEBUG [RS:1;jenkins-hbase4:42073] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-12-10 06:55:54,928 DEBUG [RS:0;jenkins-hbase4:37997] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-12-10 06:55:54,928 DEBUG [RS:1;jenkins-hbase4:42073] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-12-10 06:55:54,928 DEBUG [RS:0;jenkins-hbase4:37997] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-12-10 06:55:54,928 DEBUG [RS:1;jenkins-hbase4:42073] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-12-10 06:55:54,928 DEBUG [RS:0;jenkins-hbase4:37997] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-12-10 06:55:54,928 DEBUG [RS:1;jenkins-hbase4:42073] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-12-10 06:55:54,928 DEBUG [RS:0;jenkins-hbase4:37997] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-12-10 06:55:54,928 DEBUG [RS:1;jenkins-hbase4:42073] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-12-10 06:55:54,928 DEBUG [RS:0;jenkins-hbase4:37997] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-12-10 06:55:54,928 DEBUG [RS:1;jenkins-hbase4:42073] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-12-10 06:55:54,929 DEBUG [RS:0;jenkins-hbase4:37997] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-12-10 06:55:54,930 INFO [RS:1;jenkins-hbase4:42073] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-12-10 06:55:54,930 INFO [RS:1;jenkins-hbase4:42073] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-12-10 06:55:54,930 INFO [RS:1;jenkins-hbase4:42073] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-12-10 06:55:54,930 INFO [RS:1;jenkins-hbase4:42073] hbase.ChoreService(166): Chore ScheduledChore name=FileSystemUtilizationChore, period=300000, unit=MILLISECONDS is enabled. 2023-12-10 06:55:54,930 INFO [RS:0;jenkins-hbase4:37997] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-12-10 06:55:54,930 INFO [RS:0;jenkins-hbase4:37997] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-12-10 06:55:54,930 INFO [RS:0;jenkins-hbase4:37997] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-12-10 06:55:54,930 INFO [RS:0;jenkins-hbase4:37997] hbase.ChoreService(166): Chore ScheduledChore name=FileSystemUtilizationChore, period=300000, unit=MILLISECONDS is enabled. 2023-12-10 06:55:54,950 INFO [RS:1;jenkins-hbase4:42073] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-12-10 06:55:54,950 INFO [RS:0;jenkins-hbase4:37997] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-12-10 06:55:54,953 INFO [RS:1;jenkins-hbase4:42073] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,42073,1702191353745-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-12-10 06:55:54,953 INFO [RS:0;jenkins-hbase4:37997] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37997,1702191353702-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-12-10 06:55:54,973 INFO [RS:1;jenkins-hbase4:42073] regionserver.Replication(203): jenkins-hbase4.apache.org,42073,1702191353745 started 2023-12-10 06:55:54,973 INFO [RS:0;jenkins-hbase4:37997] regionserver.Replication(203): jenkins-hbase4.apache.org,37997,1702191353702 started 2023-12-10 06:55:54,973 INFO [RS:1;jenkins-hbase4:42073] regionserver.HRegionServer(1640): Serving as jenkins-hbase4.apache.org,42073,1702191353745, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:42073, sessionid=0x1007092abbd0002 2023-12-10 06:55:54,973 INFO [RS:0;jenkins-hbase4:37997] regionserver.HRegionServer(1640): Serving as jenkins-hbase4.apache.org,37997,1702191353702, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:37997, sessionid=0x1007092abbd0001 2023-12-10 06:55:54,973 DEBUG [RS:0;jenkins-hbase4:37997] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-12-10 06:55:54,973 DEBUG [RS:1;jenkins-hbase4:42073] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-12-10 06:55:54,973 DEBUG [RS:0;jenkins-hbase4:37997] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 06:55:54,973 DEBUG [RS:1;jenkins-hbase4:42073] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,42073,1702191353745 2023-12-10 06:55:54,974 DEBUG [RS:0;jenkins-hbase4:37997] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,37997,1702191353702' 2023-12-10 06:55:54,974 DEBUG [RS:1;jenkins-hbase4:42073] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42073,1702191353745' 2023-12-10 06:55:54,974 DEBUG [RS:1;jenkins-hbase4:42073] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-12-10 06:55:54,974 DEBUG [RS:0;jenkins-hbase4:37997] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-12-10 06:55:54,975 DEBUG [RS:1;jenkins-hbase4:42073] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-12-10 06:55:54,975 DEBUG [RS:0;jenkins-hbase4:37997] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-12-10 06:55:54,976 DEBUG [RS:1;jenkins-hbase4:42073] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-12-10 06:55:54,976 DEBUG [RS:0;jenkins-hbase4:37997] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-12-10 06:55:54,976 DEBUG [RS:1;jenkins-hbase4:42073] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-12-10 06:55:54,976 DEBUG [RS:0;jenkins-hbase4:37997] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-12-10 06:55:54,976 DEBUG [RS:1;jenkins-hbase4:42073] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,42073,1702191353745 2023-12-10 06:55:54,976 DEBUG [RS:0;jenkins-hbase4:37997] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 06:55:54,976 DEBUG [RS:0;jenkins-hbase4:37997] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,37997,1702191353702' 2023-12-10 06:55:54,976 DEBUG [RS:0;jenkins-hbase4:37997] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-12-10 06:55:54,976 DEBUG [RS:1;jenkins-hbase4:42073] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,42073,1702191353745' 2023-12-10 06:55:54,977 DEBUG [RS:1;jenkins-hbase4:42073] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-12-10 06:55:54,977 DEBUG [RS:0;jenkins-hbase4:37997] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-12-10 06:55:54,977 DEBUG [RS:1;jenkins-hbase4:42073] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-12-10 06:55:54,978 DEBUG [RS:0;jenkins-hbase4:37997] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-12-10 06:55:54,978 INFO [RS:0;jenkins-hbase4:37997] quotas.RegionServerRpcQuotaManager(67): Initializing RPC quota support 2023-12-10 06:55:54,978 DEBUG [RS:1;jenkins-hbase4:42073] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-12-10 06:55:54,978 INFO [RS:1;jenkins-hbase4:42073] quotas.RegionServerRpcQuotaManager(67): Initializing RPC quota support 2023-12-10 06:55:54,980 INFO [RS:0;jenkins-hbase4:37997] hbase.ChoreService(166): Chore ScheduledChore name=QuotaRefresherChore, period=1800000, unit=MILLISECONDS is enabled. 2023-12-10 06:55:54,980 INFO [RS:1;jenkins-hbase4:42073] hbase.ChoreService(166): Chore ScheduledChore name=QuotaRefresherChore, period=1800000, unit=MILLISECONDS is enabled. 2023-12-10 06:55:54,980 DEBUG [RS:0;jenkins-hbase4:37997] zookeeper.ZKUtil(399): regionserver:37997-0x1007092abbd0001, quorum=127.0.0.1:51670, baseZNode=/hbase Unable to get data of znode /hbase/rpc-throttle because node does not exist (not an error) 2023-12-10 06:55:54,980 DEBUG [RS:1;jenkins-hbase4:42073] zookeeper.ZKUtil(399): regionserver:42073-0x1007092abbd0002, quorum=127.0.0.1:51670, baseZNode=/hbase Unable to get data of znode /hbase/rpc-throttle because node does not exist (not an error) 2023-12-10 06:55:54,980 INFO [RS:0;jenkins-hbase4:37997] quotas.RegionServerRpcQuotaManager(73): Start rpc quota manager and rpc throttle enabled is true 2023-12-10 06:55:54,980 INFO [RS:1;jenkins-hbase4:42073] quotas.RegionServerRpcQuotaManager(73): Start rpc quota manager and rpc throttle enabled is true 2023-12-10 06:55:54,981 INFO [RS:0;jenkins-hbase4:37997] hbase.ChoreService(166): Chore ScheduledChore name=SpaceQuotaRefresherChore, period=60000, unit=MILLISECONDS is enabled. 2023-12-10 06:55:54,981 INFO [RS:1;jenkins-hbase4:42073] hbase.ChoreService(166): Chore ScheduledChore name=SpaceQuotaRefresherChore, period=60000, unit=MILLISECONDS is enabled. 2023-12-10 06:55:54,981 INFO [RS:0;jenkins-hbase4:37997] hbase.ChoreService(166): Chore ScheduledChore name=RegionSizeReportingChore, period=60000, unit=MILLISECONDS is enabled. 2023-12-10 06:55:54,981 INFO [RS:1;jenkins-hbase4:42073] hbase.ChoreService(166): Chore ScheduledChore name=RegionSizeReportingChore, period=60000, unit=MILLISECONDS is enabled. 2023-12-10 06:55:55,057 DEBUG [jenkins-hbase4:40533] assignment.AssignmentManager(2186): Processing assignQueue; systemServersCount=2, allServersCount=2 2023-12-10 06:55:55,060 DEBUG [jenkins-hbase4:40533] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-12-10 06:55:55,066 DEBUG [jenkins-hbase4:40533] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-12-10 06:55:55,066 DEBUG [jenkins-hbase4:40533] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-12-10 06:55:55,066 DEBUG [jenkins-hbase4:40533] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-12-10 06:55:55,069 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,37997,1702191353702, state=OPENING 2023-12-10 06:55:55,076 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-12-10 06:55:55,079 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-12-10 06:55:55,080 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-12-10 06:55:55,083 INFO [PEWorker-3] procedure2.ProcedureExecutor(1796): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,37997,1702191353702}] 2023-12-10 06:55:55,096 INFO [RS:0;jenkins-hbase4:37997] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37997%2C1702191353702, suffix=, logDir=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/WALs/jenkins-hbase4.apache.org,37997,1702191353702, archiveDir=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/oldWALs, maxLogs=32 2023-12-10 06:55:55,096 INFO [RS:1;jenkins-hbase4:42073] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C42073%2C1702191353745, suffix=, logDir=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/WALs/jenkins-hbase4.apache.org,42073,1702191353745, archiveDir=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/oldWALs, maxLogs=32 2023-12-10 06:55:55,119 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37723,DS-36bf9369-3517-4182-8ab9-3e7ad5cf4619,DISK] 2023-12-10 06:55:55,119 DEBUG [RS-EventLoopGroup-4-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33127,DS-966649dc-c536-4785-ba4f-0358770eeb5a,DISK] 2023-12-10 06:55:55,122 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33127,DS-966649dc-c536-4785-ba4f-0358770eeb5a,DISK] 2023-12-10 06:55:55,122 DEBUG [RS-EventLoopGroup-4-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37723,DS-36bf9369-3517-4182-8ab9-3e7ad5cf4619,DISK] 2023-12-10 06:55:55,148 INFO [RS:0;jenkins-hbase4:37997] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/WALs/jenkins-hbase4.apache.org,37997,1702191353702/jenkins-hbase4.apache.org%2C37997%2C1702191353702.1702191355101 2023-12-10 06:55:55,148 INFO [RS:1;jenkins-hbase4:42073] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/WALs/jenkins-hbase4.apache.org,42073,1702191353745/jenkins-hbase4.apache.org%2C42073%2C1702191353745.1702191355101 2023-12-10 06:55:55,148 DEBUG [RS:0;jenkins-hbase4:37997] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:33127,DS-966649dc-c536-4785-ba4f-0358770eeb5a,DISK], DatanodeInfoWithStorage[127.0.0.1:37723,DS-36bf9369-3517-4182-8ab9-3e7ad5cf4619,DISK]] 2023-12-10 06:55:55,149 DEBUG [RS:1;jenkins-hbase4:42073] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37723,DS-36bf9369-3517-4182-8ab9-3e7ad5cf4619,DISK], DatanodeInfoWithStorage[127.0.0.1:33127,DS-966649dc-c536-4785-ba4f-0358770eeb5a,DISK]] 2023-12-10 06:55:55,271 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 06:55:55,273 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=AdminService, sasl=false 2023-12-10 06:55:55,276 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41676, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-12-10 06:55:55,288 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-12-10 06:55:55,288 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-12-10 06:55:55,292 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37997%2C1702191353702.meta, suffix=.meta, logDir=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/WALs/jenkins-hbase4.apache.org,37997,1702191353702, archiveDir=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/oldWALs, maxLogs=32 2023-12-10 06:55:55,308 DEBUG [RS-EventLoopGroup-4-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37723,DS-36bf9369-3517-4182-8ab9-3e7ad5cf4619,DISK] 2023-12-10 06:55:55,309 DEBUG [RS-EventLoopGroup-4-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33127,DS-966649dc-c536-4785-ba4f-0358770eeb5a,DISK] 2023-12-10 06:55:55,315 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/WALs/jenkins-hbase4.apache.org,37997,1702191353702/jenkins-hbase4.apache.org%2C37997%2C1702191353702.meta.1702191355293.meta 2023-12-10 06:55:55,318 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37723,DS-36bf9369-3517-4182-8ab9-3e7ad5cf4619,DISK], DatanodeInfoWithStorage[127.0.0.1:33127,DS-966649dc-c536-4785-ba4f-0358770eeb5a,DISK]] 2023-12-10 06:55:55,319 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-12-10 06:55:55,320 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-12-10 06:55:55,336 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-12-10 06:55:55,340 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-12-10 06:55:55,345 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-12-10 06:55:55,345 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-12-10 06:55:55,345 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-12-10 06:55:55,345 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-12-10 06:55:55,348 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-12-10 06:55:55,350 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740/info 2023-12-10 06:55:55,350 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740/info 2023-12-10 06:55:55,351 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:10, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-12-10 06:55:55,352 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-12-10 06:55:55,352 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-12-10 06:55:55,353 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740/rep_barrier 2023-12-10 06:55:55,353 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740/rep_barrier 2023-12-10 06:55:55,354 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:10, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-12-10 06:55:55,355 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-12-10 06:55:55,355 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-12-10 06:55:55,356 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740/table 2023-12-10 06:55:55,356 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740/table 2023-12-10 06:55:55,357 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:10, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-12-10 06:55:55,357 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-12-10 06:55:55,359 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740 2023-12-10 06:55:55,362 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740 2023-12-10 06:55:55,365 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-12-10 06:55:55,368 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-12-10 06:55:55,369 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62409337, jitterRate=-0.0700284093618393}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-12-10 06:55:55,370 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-12-10 06:55:55,381 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1702191355262 2023-12-10 06:55:55,398 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for hbase:meta,,1.1588230740 2023-12-10 06:55:55,399 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-12-10 06:55:55,399 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,37997,1702191353702, state=OPEN 2023-12-10 06:55:55,403 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-12-10 06:55:55,403 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-12-10 06:55:55,407 INFO [PEWorker-5] procedure2.ProcedureExecutor(1953): Finished subprocedure pid=3, resume processing ppid=2 2023-12-10 06:55:55,407 INFO [PEWorker-5] procedure2.ProcedureExecutor(1457): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,37997,1702191353702 in 320 msec 2023-12-10 06:55:55,414 INFO [PEWorker-1] procedure2.ProcedureExecutor(1953): Finished subprocedure pid=2, resume processing ppid=1 2023-12-10 06:55:55,414 INFO [PEWorker-1] procedure2.ProcedureExecutor(1457): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 517 msec 2023-12-10 06:55:55,419 INFO [PEWorker-2] procedure2.ProcedureExecutor(1457): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 760 msec 2023-12-10 06:55:55,419 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(954): Wait for region servers to report in: status=null, state=RUNNING, startTime=1702191355419, completionTime=-1 2023-12-10 06:55:55,419 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(808): Finished waiting on RegionServer count=2; waited=0ms, expected min=2 server(s), max=2 server(s), master is running 2023-12-10 06:55:55,420 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1527): Joining cluster... 2023-12-10 06:55:55,480 DEBUG [hconnection-0x2ae805ba-shared-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-12-10 06:55:55,482 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41678, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-12-10 06:55:55,498 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1539): Number of RegionServers=2 2023-12-10 06:55:55,498 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1702191415498 2023-12-10 06:55:55,498 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1702191475498 2023-12-10 06:55:55,498 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1546): Joined the cluster in 78 msec 2023-12-10 06:55:55,519 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40533,1702191352570-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-12-10 06:55:55,519 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40533,1702191352570-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-12-10 06:55:55,520 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40533,1702191352570-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-12-10 06:55:55,521 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:40533, period=300000, unit=MILLISECONDS is enabled. 2023-12-10 06:55:55,522 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-12-10 06:55:55,527 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-12-10 06:55:55,535 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-12-10 06:55:55,536 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2154): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-12-10 06:55:55,545 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1075): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-12-10 06:55:55,547 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-12-10 06:55:55,550 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-12-10 06:55:55,570 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/hbase/namespace/3b6714111b5dc2b8d14208f4c54166e3 2023-12-10 06:55:55,572 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/hbase/namespace/3b6714111b5dc2b8d14208f4c54166e3 empty. 2023-12-10 06:55:55,573 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/hbase/namespace/3b6714111b5dc2b8d14208f4c54166e3 2023-12-10 06:55:55,573 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-12-10 06:55:55,603 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-12-10 06:55:55,605 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 3b6714111b5dc2b8d14208f4c54166e3, NAME => 'hbase:namespace,,1702191355536.3b6714111b5dc2b8d14208f4c54166e3.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp 2023-12-10 06:55:55,622 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1702191355536.3b6714111b5dc2b8d14208f4c54166e3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-12-10 06:55:55,622 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 3b6714111b5dc2b8d14208f4c54166e3, disabling compactions & flushes 2023-12-10 06:55:55,622 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1702191355536.3b6714111b5dc2b8d14208f4c54166e3. 2023-12-10 06:55:55,622 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1702191355536.3b6714111b5dc2b8d14208f4c54166e3. 2023-12-10 06:55:55,622 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1702191355536.3b6714111b5dc2b8d14208f4c54166e3. after waiting 0 ms 2023-12-10 06:55:55,622 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1702191355536.3b6714111b5dc2b8d14208f4c54166e3. 2023-12-10 06:55:55,622 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1702191355536.3b6714111b5dc2b8d14208f4c54166e3. 2023-12-10 06:55:55,622 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 3b6714111b5dc2b8d14208f4c54166e3: 2023-12-10 06:55:55,626 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-12-10 06:55:55,641 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1702191355536.3b6714111b5dc2b8d14208f4c54166e3.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1702191355629"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1702191355629"}]},"ts":"1702191355629"} 2023-12-10 06:55:55,665 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-12-10 06:55:55,667 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-12-10 06:55:55,671 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1702191355667"}]},"ts":"1702191355667"} 2023-12-10 06:55:55,677 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-12-10 06:55:55,684 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-12-10 06:55:55,685 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-12-10 06:55:55,685 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-12-10 06:55:55,685 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-12-10 06:55:55,688 INFO [PEWorker-3] procedure2.ProcedureExecutor(1796): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=3b6714111b5dc2b8d14208f4c54166e3, ASSIGN}] 2023-12-10 06:55:55,690 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=3b6714111b5dc2b8d14208f4c54166e3, ASSIGN 2023-12-10 06:55:55,692 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=3b6714111b5dc2b8d14208f4c54166e3, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37997,1702191353702; forceNewPlan=false, retain=false 2023-12-10 06:55:55,845 INFO [jenkins-hbase4:40533] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-12-10 06:55:55,846 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=3b6714111b5dc2b8d14208f4c54166e3, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 06:55:55,846 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1702191355536.3b6714111b5dc2b8d14208f4c54166e3.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1702191355845"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1702191355845"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1702191355845"}]},"ts":"1702191355845"} 2023-12-10 06:55:55,851 INFO [PEWorker-5] procedure2.ProcedureExecutor(1796): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure 3b6714111b5dc2b8d14208f4c54166e3, server=jenkins-hbase4.apache.org,37997,1702191353702}] 2023-12-10 06:55:56,005 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 06:55:56,012 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1702191355536.3b6714111b5dc2b8d14208f4c54166e3. 2023-12-10 06:55:56,012 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 3b6714111b5dc2b8d14208f4c54166e3, NAME => 'hbase:namespace,,1702191355536.3b6714111b5dc2b8d14208f4c54166e3.', STARTKEY => '', ENDKEY => ''} 2023-12-10 06:55:56,013 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 3b6714111b5dc2b8d14208f4c54166e3 2023-12-10 06:55:56,013 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1702191355536.3b6714111b5dc2b8d14208f4c54166e3.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-12-10 06:55:56,013 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 3b6714111b5dc2b8d14208f4c54166e3 2023-12-10 06:55:56,013 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 3b6714111b5dc2b8d14208f4c54166e3 2023-12-10 06:55:56,016 INFO [StoreOpener-3b6714111b5dc2b8d14208f4c54166e3-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 3b6714111b5dc2b8d14208f4c54166e3 2023-12-10 06:55:56,018 DEBUG [StoreOpener-3b6714111b5dc2b8d14208f4c54166e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/namespace/3b6714111b5dc2b8d14208f4c54166e3/info 2023-12-10 06:55:56,018 DEBUG [StoreOpener-3b6714111b5dc2b8d14208f4c54166e3-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/namespace/3b6714111b5dc2b8d14208f4c54166e3/info 2023-12-10 06:55:56,019 INFO [StoreOpener-3b6714111b5dc2b8d14208f4c54166e3-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:10, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 3b6714111b5dc2b8d14208f4c54166e3 columnFamilyName info 2023-12-10 06:55:56,019 INFO [StoreOpener-3b6714111b5dc2b8d14208f4c54166e3-1] regionserver.HStore(310): Store=3b6714111b5dc2b8d14208f4c54166e3/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-12-10 06:55:56,021 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/namespace/3b6714111b5dc2b8d14208f4c54166e3 2023-12-10 06:55:56,022 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/namespace/3b6714111b5dc2b8d14208f4c54166e3 2023-12-10 06:55:56,027 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 3b6714111b5dc2b8d14208f4c54166e3 2023-12-10 06:55:56,030 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/namespace/3b6714111b5dc2b8d14208f4c54166e3/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-12-10 06:55:56,031 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 3b6714111b5dc2b8d14208f4c54166e3; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63020316, jitterRate=-0.06092411279678345}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-12-10 06:55:56,031 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 3b6714111b5dc2b8d14208f4c54166e3: 2023-12-10 06:55:56,033 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for hbase:namespace,,1702191355536.3b6714111b5dc2b8d14208f4c54166e3., pid=6, masterSystemTime=1702191356005 2023-12-10 06:55:56,037 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for hbase:namespace,,1702191355536.3b6714111b5dc2b8d14208f4c54166e3. 2023-12-10 06:55:56,037 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1702191355536.3b6714111b5dc2b8d14208f4c54166e3. 2023-12-10 06:55:56,038 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=3b6714111b5dc2b8d14208f4c54166e3, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 06:55:56,038 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1702191355536.3b6714111b5dc2b8d14208f4c54166e3.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1702191356038"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1702191356038"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1702191356038"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1702191356038"}]},"ts":"1702191356038"} 2023-12-10 06:55:56,045 INFO [PEWorker-2] procedure2.ProcedureExecutor(1953): Finished subprocedure pid=6, resume processing ppid=5 2023-12-10 06:55:56,046 INFO [PEWorker-2] procedure2.ProcedureExecutor(1457): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure 3b6714111b5dc2b8d14208f4c54166e3, server=jenkins-hbase4.apache.org,37997,1702191353702 in 191 msec 2023-12-10 06:55:56,049 INFO [PEWorker-3] procedure2.ProcedureExecutor(1953): Finished subprocedure pid=5, resume processing ppid=4 2023-12-10 06:55:56,049 INFO [PEWorker-3] procedure2.ProcedureExecutor(1457): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=3b6714111b5dc2b8d14208f4c54166e3, ASSIGN in 357 msec 2023-12-10 06:55:56,051 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-12-10 06:55:56,051 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1702191356051"}]},"ts":"1702191356051"} 2023-12-10 06:55:56,053 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-12-10 06:55:56,057 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-12-10 06:55:56,060 INFO [PEWorker-4] procedure2.ProcedureExecutor(1457): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 520 msec 2023-12-10 06:55:56,148 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(165): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-12-10 06:55:56,150 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-12-10 06:55:56,150 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-12-10 06:55:56,181 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1075): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-12-10 06:55:56,199 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-12-10 06:55:56,206 INFO [PEWorker-5] procedure2.ProcedureExecutor(1457): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 30 msec 2023-12-10 06:55:56,215 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1075): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-12-10 06:55:56,226 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-12-10 06:55:56,231 INFO [PEWorker-1] procedure2.ProcedureExecutor(1457): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 15 msec 2023-12-10 06:55:56,241 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-12-10 06:55:56,244 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-12-10 06:55:56,244 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1084): Master has completed initialization 2.440sec 2023-12-10 06:55:56,247 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(103): Quota table not found. Creating... 2023-12-10 06:55:56,247 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2154): Client=null/null create 'hbase:quota', {NAME => 'q', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'u', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-12-10 06:55:56,248 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1075): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:quota 2023-12-10 06:55:56,248 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(107): Initializing quota support 2023-12-10 06:55:56,250 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:quota execute state=CREATE_TABLE_PRE_OPERATION 2023-12-10 06:55:56,250 INFO [master/jenkins-hbase4:0:becomeActiveMaster] namespace.NamespaceStateManager(59): Namespace State Manager started. 2023-12-10 06:55:56,252 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:quota execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-12-10 06:55:56,254 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b 2023-12-10 06:55:56,255 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b empty. 2023-12-10 06:55:56,257 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b 2023-12-10 06:55:56,257 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived hbase:quota regions 2023-12-10 06:55:56,258 INFO [master/jenkins-hbase4:0:becomeActiveMaster] namespace.NamespaceStateManager(222): Finished updating state of 2 namespaces. 2023-12-10 06:55:56,258 INFO [master/jenkins-hbase4:0:becomeActiveMaster] namespace.NamespaceAuditor(50): NamespaceAuditor started. 2023-12-10 06:55:56,259 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=QuotaObserverChore, period=60000, unit=MILLISECONDS is enabled. 2023-12-10 06:55:56,260 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=QuotaObserverChore, period=300000, unit=MILLISECONDS is enabled. 2023-12-10 06:55:56,261 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-12-10 06:55:56,262 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-12-10 06:55:56,263 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40533,1702191352570-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-12-10 06:55:56,264 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40533,1702191352570-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-12-10 06:55:56,274 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1176): Balancer post startup initialization complete, took 0 seconds 2023-12-10 06:55:56,274 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,40533,1702191352570-OldWALsDirSizeChore, period=300000, unit=MILLISECONDS is enabled. 2023-12-10 06:55:56,279 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/hbase/quota/.tabledesc/.tableinfo.0000000001 2023-12-10 06:55:56,280 DEBUG [Listener at localhost/38369] zookeeper.ReadOnlyZKClient(139): Connect 0x11b46918 to 127.0.0.1:51670 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-12-10 06:55:56,281 INFO [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(7675): creating {ENCODED => dca2e6ecbeef49fb8929fd36dd74df8b, NAME => 'hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:quota', {NAME => 'q', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'u', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp 2023-12-10 06:55:56,290 DEBUG [Listener at localhost/38369] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@58d4a0dc, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-12-10 06:55:56,304 DEBUG [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(866): Instantiated hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-12-10 06:55:56,305 DEBUG [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(1604): Closing dca2e6ecbeef49fb8929fd36dd74df8b, disabling compactions & flushes 2023-12-10 06:55:56,305 INFO [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(1626): Closing region hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b. 2023-12-10 06:55:56,305 DEBUG [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b. 2023-12-10 06:55:56,305 DEBUG [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b. after waiting 0 ms 2023-12-10 06:55:56,305 DEBUG [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b. 2023-12-10 06:55:56,305 INFO [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(1838): Closed hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b. 2023-12-10 06:55:56,305 DEBUG [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(1558): Region close journal for dca2e6ecbeef49fb8929fd36dd74df8b: 2023-12-10 06:55:56,310 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:quota execute state=CREATE_TABLE_ADD_TO_META 2023-12-10 06:55:56,311 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b.","families":{"info":[{"qualifier":"regioninfo","vlen":37,"tag":[],"timestamp":"1702191356311"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1702191356311"}]},"ts":"1702191356311"} 2023-12-10 06:55:56,311 DEBUG [hconnection-0x505566b3-shared-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-12-10 06:55:56,314 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-12-10 06:55:56,316 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:quota execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-12-10 06:55:56,316 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:quota","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1702191356316"}]},"ts":"1702191356316"} 2023-12-10 06:55:56,318 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=hbase:quota, state=ENABLING in hbase:meta 2023-12-10 06:55:56,323 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41680, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-12-10 06:55:56,323 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-12-10 06:55:56,324 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-12-10 06:55:56,325 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-12-10 06:55:56,325 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-12-10 06:55:56,325 INFO [PEWorker-2] procedure2.ProcedureExecutor(1796): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:quota, region=dca2e6ecbeef49fb8929fd36dd74df8b, ASSIGN}] 2023-12-10 06:55:56,326 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:quota, region=dca2e6ecbeef49fb8929fd36dd74df8b, ASSIGN 2023-12-10 06:55:56,328 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:quota, region=dca2e6ecbeef49fb8929fd36dd74df8b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37997,1702191353702; forceNewPlan=false, retain=false 2023-12-10 06:55:56,333 INFO [Listener at localhost/38369] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,40533,1702191352570 2023-12-10 06:55:56,334 INFO [Listener at localhost/38369] hbase.Waiter(180): Waiting up to [30,000] milli-secs(wait.for.ratio=[1]) 2023-12-10 06:55:56,337 DEBUG [Listener at localhost/38369] client.ConnectionImplementation(716): Table hbase:quota not enabled 2023-12-10 06:55:56,439 DEBUG [Listener at localhost/38369] client.ConnectionImplementation(716): Table hbase:quota not enabled 2023-12-10 06:55:56,478 INFO [jenkins-hbase4:40533] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-12-10 06:55:56,479 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=dca2e6ecbeef49fb8929fd36dd74df8b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 06:55:56,479 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b.","families":{"info":[{"qualifier":"regioninfo","vlen":37,"tag":[],"timestamp":"1702191356479"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1702191356479"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1702191356479"}]},"ts":"1702191356479"} 2023-12-10 06:55:56,482 INFO [PEWorker-4] procedure2.ProcedureExecutor(1796): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure dca2e6ecbeef49fb8929fd36dd74df8b, server=jenkins-hbase4.apache.org,37997,1702191353702}] 2023-12-10 06:55:56,541 DEBUG [Listener at localhost/38369] client.ConnectionImplementation(716): Table hbase:quota not enabled 2023-12-10 06:55:56,635 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 06:55:56,641 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b. 2023-12-10 06:55:56,642 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => dca2e6ecbeef49fb8929fd36dd74df8b, NAME => 'hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b.', STARTKEY => '', ENDKEY => ''} 2023-12-10 06:55:56,642 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table quota dca2e6ecbeef49fb8929fd36dd74df8b 2023-12-10 06:55:56,642 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-12-10 06:55:56,642 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for dca2e6ecbeef49fb8929fd36dd74df8b 2023-12-10 06:55:56,642 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for dca2e6ecbeef49fb8929fd36dd74df8b 2023-12-10 06:55:56,644 DEBUG [Listener at localhost/38369] client.ConnectionImplementation(716): Table hbase:quota not enabled 2023-12-10 06:55:56,645 INFO [StoreOpener-dca2e6ecbeef49fb8929fd36dd74df8b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family q of region dca2e6ecbeef49fb8929fd36dd74df8b 2023-12-10 06:55:56,647 DEBUG [StoreOpener-dca2e6ecbeef49fb8929fd36dd74df8b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/q 2023-12-10 06:55:56,647 DEBUG [StoreOpener-dca2e6ecbeef49fb8929fd36dd74df8b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/q 2023-12-10 06:55:56,648 INFO [StoreOpener-dca2e6ecbeef49fb8929fd36dd74df8b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:10, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dca2e6ecbeef49fb8929fd36dd74df8b columnFamilyName q 2023-12-10 06:55:56,648 INFO [StoreOpener-dca2e6ecbeef49fb8929fd36dd74df8b-1] regionserver.HStore(310): Store=dca2e6ecbeef49fb8929fd36dd74df8b/q, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-12-10 06:55:56,649 INFO [StoreOpener-dca2e6ecbeef49fb8929fd36dd74df8b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family u of region dca2e6ecbeef49fb8929fd36dd74df8b 2023-12-10 06:55:56,651 DEBUG [StoreOpener-dca2e6ecbeef49fb8929fd36dd74df8b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/u 2023-12-10 06:55:56,651 DEBUG [StoreOpener-dca2e6ecbeef49fb8929fd36dd74df8b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/u 2023-12-10 06:55:56,651 INFO [StoreOpener-dca2e6ecbeef49fb8929fd36dd74df8b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:10, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dca2e6ecbeef49fb8929fd36dd74df8b columnFamilyName u 2023-12-10 06:55:56,652 INFO [StoreOpener-dca2e6ecbeef49fb8929fd36dd74df8b-1] regionserver.HStore(310): Store=dca2e6ecbeef49fb8929fd36dd74df8b/u, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-12-10 06:55:56,653 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b 2023-12-10 06:55:56,654 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b 2023-12-10 06:55:56,657 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:quota descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-12-10 06:55:56,659 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for dca2e6ecbeef49fb8929fd36dd74df8b 2023-12-10 06:55:56,662 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-12-10 06:55:56,663 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened dca2e6ecbeef49fb8929fd36dd74df8b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65526372, jitterRate=-0.023580968379974365}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-12-10 06:55:56,663 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for dca2e6ecbeef49fb8929fd36dd74df8b: 2023-12-10 06:55:56,664 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., pid=11, masterSystemTime=1702191356635 2023-12-10 06:55:56,666 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b. 2023-12-10 06:55:56,667 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b. 2023-12-10 06:55:56,667 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=dca2e6ecbeef49fb8929fd36dd74df8b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 06:55:56,668 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b.","families":{"info":[{"qualifier":"regioninfo","vlen":37,"tag":[],"timestamp":"1702191356667"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1702191356667"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1702191356667"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1702191356667"}]},"ts":"1702191356667"} 2023-12-10 06:55:56,673 INFO [PEWorker-1] procedure2.ProcedureExecutor(1953): Finished subprocedure pid=11, resume processing ppid=10 2023-12-10 06:55:56,673 INFO [PEWorker-1] procedure2.ProcedureExecutor(1457): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure dca2e6ecbeef49fb8929fd36dd74df8b, server=jenkins-hbase4.apache.org,37997,1702191353702 in 188 msec 2023-12-10 06:55:56,676 INFO [PEWorker-2] procedure2.ProcedureExecutor(1953): Finished subprocedure pid=10, resume processing ppid=9 2023-12-10 06:55:56,677 INFO [PEWorker-2] procedure2.ProcedureExecutor(1457): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=hbase:quota, region=dca2e6ecbeef49fb8929fd36dd74df8b, ASSIGN in 348 msec 2023-12-10 06:55:56,678 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:quota execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-12-10 06:55:56,678 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:quota","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1702191356678"}]},"ts":"1702191356678"} 2023-12-10 06:55:56,680 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:quota, state=ENABLED in hbase:meta 2023-12-10 06:55:56,683 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:quota execute state=CREATE_TABLE_POST_OPERATION 2023-12-10 06:55:56,685 INFO [PEWorker-3] procedure2.ProcedureExecutor(1457): Finished pid=9, state=SUCCESS; CreateTableProcedure table=hbase:quota in 436 msec 2023-12-10 06:55:56,757 DEBUG [Listener at localhost/38369] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-12-10 06:55:56,759 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41548, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-12-10 06:55:56,784 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40533] master.HMaster$4(2118): Client=jenkins//172.31.14.131 create 'TestQuotaAdmin0', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '1'}}, {NAME => 'cf', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-12-10 06:55:56,787 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40533] procedure2.ProcedureExecutor(1075): Stored pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=TestQuotaAdmin0 2023-12-10 06:55:56,789 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=TestQuotaAdmin0 execute state=CREATE_TABLE_PRE_OPERATION 2023-12-10 06:55:56,791 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=TestQuotaAdmin0 execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-12-10 06:55:56,794 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/default/TestQuotaAdmin0/738cc1ffb58c4f3e8fbde456163cdb37 2023-12-10 06:55:56,794 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40533] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "TestQuotaAdmin0" procId is: 12 2023-12-10 06:55:56,794 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/default/TestQuotaAdmin0/738cc1ffb58c4f3e8fbde456163cdb37 empty. 2023-12-10 06:55:56,795 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/default/TestQuotaAdmin0/738cc1ffb58c4f3e8fbde456163cdb37 2023-12-10 06:55:56,795 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived TestQuotaAdmin0 regions 2023-12-10 06:55:56,802 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40533] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-12-10 06:55:56,812 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/default/TestQuotaAdmin0/.tabledesc/.tableinfo.0000000001 2023-12-10 06:55:56,814 INFO [RegionOpenAndInit-TestQuotaAdmin0-pool-0] regionserver.HRegion(7675): creating {ENCODED => 738cc1ffb58c4f3e8fbde456163cdb37, NAME => 'TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='TestQuotaAdmin0', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '1'}}, {NAME => 'cf', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp 2023-12-10 06:55:56,829 DEBUG [RegionOpenAndInit-TestQuotaAdmin0-pool-0] regionserver.HRegion(866): Instantiated TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-12-10 06:55:56,829 DEBUG [RegionOpenAndInit-TestQuotaAdmin0-pool-0] regionserver.HRegion(1604): Closing 738cc1ffb58c4f3e8fbde456163cdb37, disabling compactions & flushes 2023-12-10 06:55:56,829 INFO [RegionOpenAndInit-TestQuotaAdmin0-pool-0] regionserver.HRegion(1626): Closing region TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37. 2023-12-10 06:55:56,829 DEBUG [RegionOpenAndInit-TestQuotaAdmin0-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37. 2023-12-10 06:55:56,829 DEBUG [RegionOpenAndInit-TestQuotaAdmin0-pool-0] regionserver.HRegion(1714): Acquired close lock on TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37. after waiting 0 ms 2023-12-10 06:55:56,829 DEBUG [RegionOpenAndInit-TestQuotaAdmin0-pool-0] regionserver.HRegion(1724): Updates disabled for region TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37. 2023-12-10 06:55:56,829 INFO [RegionOpenAndInit-TestQuotaAdmin0-pool-0] regionserver.HRegion(1838): Closed TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37. 2023-12-10 06:55:56,829 DEBUG [RegionOpenAndInit-TestQuotaAdmin0-pool-0] regionserver.HRegion(1558): Region close journal for 738cc1ffb58c4f3e8fbde456163cdb37: 2023-12-10 06:55:56,833 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=TestQuotaAdmin0 execute state=CREATE_TABLE_ADD_TO_META 2023-12-10 06:55:56,834 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1702191356834"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1702191356834"}]},"ts":"1702191356834"} 2023-12-10 06:55:56,837 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-12-10 06:55:56,838 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=TestQuotaAdmin0 execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-12-10 06:55:56,838 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestQuotaAdmin0","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1702191356838"}]},"ts":"1702191356838"} 2023-12-10 06:55:56,840 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=TestQuotaAdmin0, state=ENABLING in hbase:meta 2023-12-10 06:55:56,846 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-12-10 06:55:56,847 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-12-10 06:55:56,847 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-12-10 06:55:56,847 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-12-10 06:55:56,847 INFO [PEWorker-4] procedure2.ProcedureExecutor(1796): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestQuotaAdmin0, region=738cc1ffb58c4f3e8fbde456163cdb37, ASSIGN}] 2023-12-10 06:55:56,848 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestQuotaAdmin0, region=738cc1ffb58c4f3e8fbde456163cdb37, ASSIGN 2023-12-10 06:55:56,850 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestQuotaAdmin0, region=738cc1ffb58c4f3e8fbde456163cdb37, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37997,1702191353702; forceNewPlan=false, retain=false 2023-12-10 06:55:57,000 INFO [jenkins-hbase4:40533] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-12-10 06:55:57,001 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=738cc1ffb58c4f3e8fbde456163cdb37, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 06:55:57,001 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1702191357001"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1702191357001"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1702191357001"}]},"ts":"1702191357001"} 2023-12-10 06:55:57,004 INFO [PEWorker-1] procedure2.ProcedureExecutor(1796): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; OpenRegionProcedure 738cc1ffb58c4f3e8fbde456163cdb37, server=jenkins-hbase4.apache.org,37997,1702191353702}] 2023-12-10 06:55:57,056 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40533] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-12-10 06:55:57,156 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 06:55:57,162 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37. 2023-12-10 06:55:57,162 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 738cc1ffb58c4f3e8fbde456163cdb37, NAME => 'TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37.', STARTKEY => '', ENDKEY => ''} 2023-12-10 06:55:57,163 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestQuotaAdmin0 738cc1ffb58c4f3e8fbde456163cdb37 2023-12-10 06:55:57,163 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-12-10 06:55:57,163 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 738cc1ffb58c4f3e8fbde456163cdb37 2023-12-10 06:55:57,163 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 738cc1ffb58c4f3e8fbde456163cdb37 2023-12-10 06:55:57,165 INFO [StoreOpener-738cc1ffb58c4f3e8fbde456163cdb37-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf of region 738cc1ffb58c4f3e8fbde456163cdb37 2023-12-10 06:55:57,167 DEBUG [StoreOpener-738cc1ffb58c4f3e8fbde456163cdb37-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin0/738cc1ffb58c4f3e8fbde456163cdb37/cf 2023-12-10 06:55:57,167 DEBUG [StoreOpener-738cc1ffb58c4f3e8fbde456163cdb37-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin0/738cc1ffb58c4f3e8fbde456163cdb37/cf 2023-12-10 06:55:57,167 INFO [StoreOpener-738cc1ffb58c4f3e8fbde456163cdb37-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:10, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 738cc1ffb58c4f3e8fbde456163cdb37 columnFamilyName cf 2023-12-10 06:55:57,168 INFO [StoreOpener-738cc1ffb58c4f3e8fbde456163cdb37-1] regionserver.HStore(310): Store=738cc1ffb58c4f3e8fbde456163cdb37/cf, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-12-10 06:55:57,170 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin0/738cc1ffb58c4f3e8fbde456163cdb37 2023-12-10 06:55:57,171 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin0/738cc1ffb58c4f3e8fbde456163cdb37 2023-12-10 06:55:57,176 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 738cc1ffb58c4f3e8fbde456163cdb37 2023-12-10 06:55:57,179 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin0/738cc1ffb58c4f3e8fbde456163cdb37/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-12-10 06:55:57,180 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 738cc1ffb58c4f3e8fbde456163cdb37; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72040703, jitterRate=0.07349012792110443}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-12-10 06:55:57,180 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 738cc1ffb58c4f3e8fbde456163cdb37: 2023-12-10 06:55:57,182 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37., pid=14, masterSystemTime=1702191357156 2023-12-10 06:55:57,184 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37. 2023-12-10 06:55:57,184 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37. 2023-12-10 06:55:57,185 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=738cc1ffb58c4f3e8fbde456163cdb37, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 06:55:57,186 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1702191357185"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1702191357185"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1702191357185"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1702191357185"}]},"ts":"1702191357185"} 2023-12-10 06:55:57,191 INFO [PEWorker-3] procedure2.ProcedureExecutor(1953): Finished subprocedure pid=14, resume processing ppid=13 2023-12-10 06:55:57,191 INFO [PEWorker-3] procedure2.ProcedureExecutor(1457): Finished pid=14, ppid=13, state=SUCCESS; OpenRegionProcedure 738cc1ffb58c4f3e8fbde456163cdb37, server=jenkins-hbase4.apache.org,37997,1702191353702 in 184 msec 2023-12-10 06:55:57,194 INFO [PEWorker-4] procedure2.ProcedureExecutor(1953): Finished subprocedure pid=13, resume processing ppid=12 2023-12-10 06:55:57,195 INFO [PEWorker-4] procedure2.ProcedureExecutor(1457): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=TestQuotaAdmin0, region=738cc1ffb58c4f3e8fbde456163cdb37, ASSIGN in 344 msec 2023-12-10 06:55:57,196 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=TestQuotaAdmin0 execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-12-10 06:55:57,196 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestQuotaAdmin0","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1702191357196"}]},"ts":"1702191357196"} 2023-12-10 06:55:57,198 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=TestQuotaAdmin0, state=ENABLED in hbase:meta 2023-12-10 06:55:57,201 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=12, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=TestQuotaAdmin0 execute state=CREATE_TABLE_POST_OPERATION 2023-12-10 06:55:57,203 INFO [PEWorker-5] procedure2.ProcedureExecutor(1457): Finished pid=12, state=SUCCESS; CreateTableProcedure table=TestQuotaAdmin0 in 416 msec 2023-12-10 06:55:57,557 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=40533] master.MasterRpcServices(1230): Checking to see if procedure is done pid=12 2023-12-10 06:55:57,558 INFO [Listener at localhost/38369] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:TestQuotaAdmin0, procId: 12 completed 2023-12-10 06:55:57,558 DEBUG [Listener at localhost/38369] hbase.HBaseTestingUtility(3430): Waiting until all regions of table TestQuotaAdmin0 get assigned. Timeout = 60000ms 2023-12-10 06:55:57,559 INFO [Listener at localhost/38369] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-12-10 06:55:57,563 INFO [Listener at localhost/38369] hbase.HBaseTestingUtility(3484): All regions for table TestQuotaAdmin0 assigned to meta. Checking AM states. 2023-12-10 06:55:57,563 INFO [Listener at localhost/38369] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-12-10 06:55:57,564 INFO [Listener at localhost/38369] hbase.HBaseTestingUtility(3504): All regions for table TestQuotaAdmin0 assigned. 2023-12-10 06:55:57,564 INFO [Listener at localhost/38369] hbase.Waiter(180): Waiting up to [30,000] milli-secs(wait.for.ratio=[1]) 2023-12-10 06:55:57,577 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-12-10 06:55:57,579 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41556, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=MasterService 2023-12-10 06:55:57,580 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.HMaster$4(2118): Client=jenkins//172.31.14.131 create 'TestQuotaAdmin1', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '1'}}, {NAME => 'cf', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-12-10 06:55:57,582 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] procedure2.ProcedureExecutor(1075): Stored pid=15, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=TestQuotaAdmin1 2023-12-10 06:55:57,584 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=15, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=TestQuotaAdmin1 execute state=CREATE_TABLE_PRE_OPERATION 2023-12-10 06:55:57,584 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "TestQuotaAdmin1" procId is: 15 2023-12-10 06:55:57,586 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=15, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=TestQuotaAdmin1 execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-12-10 06:55:57,586 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-12-10 06:55:57,588 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/default/TestQuotaAdmin1/0522cf92a3f0ad796f58de4381345d12 2023-12-10 06:55:57,589 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/default/TestQuotaAdmin1/0522cf92a3f0ad796f58de4381345d12 empty. 2023-12-10 06:55:57,589 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/default/TestQuotaAdmin1/0522cf92a3f0ad796f58de4381345d12 2023-12-10 06:55:57,589 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived TestQuotaAdmin1 regions 2023-12-10 06:55:57,609 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/default/TestQuotaAdmin1/.tabledesc/.tableinfo.0000000001 2023-12-10 06:55:57,610 INFO [RegionOpenAndInit-TestQuotaAdmin1-pool-0] regionserver.HRegion(7675): creating {ENCODED => 0522cf92a3f0ad796f58de4381345d12, NAME => 'TestQuotaAdmin1,,1702191357579.0522cf92a3f0ad796f58de4381345d12.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='TestQuotaAdmin1', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '1'}}, {NAME => 'cf', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp 2023-12-10 06:55:57,626 DEBUG [RegionOpenAndInit-TestQuotaAdmin1-pool-0] regionserver.HRegion(866): Instantiated TestQuotaAdmin1,,1702191357579.0522cf92a3f0ad796f58de4381345d12.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-12-10 06:55:57,626 DEBUG [RegionOpenAndInit-TestQuotaAdmin1-pool-0] regionserver.HRegion(1604): Closing 0522cf92a3f0ad796f58de4381345d12, disabling compactions & flushes 2023-12-10 06:55:57,626 INFO [RegionOpenAndInit-TestQuotaAdmin1-pool-0] regionserver.HRegion(1626): Closing region TestQuotaAdmin1,,1702191357579.0522cf92a3f0ad796f58de4381345d12. 2023-12-10 06:55:57,626 DEBUG [RegionOpenAndInit-TestQuotaAdmin1-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestQuotaAdmin1,,1702191357579.0522cf92a3f0ad796f58de4381345d12. 2023-12-10 06:55:57,626 DEBUG [RegionOpenAndInit-TestQuotaAdmin1-pool-0] regionserver.HRegion(1714): Acquired close lock on TestQuotaAdmin1,,1702191357579.0522cf92a3f0ad796f58de4381345d12. after waiting 0 ms 2023-12-10 06:55:57,626 DEBUG [RegionOpenAndInit-TestQuotaAdmin1-pool-0] regionserver.HRegion(1724): Updates disabled for region TestQuotaAdmin1,,1702191357579.0522cf92a3f0ad796f58de4381345d12. 2023-12-10 06:55:57,626 INFO [RegionOpenAndInit-TestQuotaAdmin1-pool-0] regionserver.HRegion(1838): Closed TestQuotaAdmin1,,1702191357579.0522cf92a3f0ad796f58de4381345d12. 2023-12-10 06:55:57,626 DEBUG [RegionOpenAndInit-TestQuotaAdmin1-pool-0] regionserver.HRegion(1558): Region close journal for 0522cf92a3f0ad796f58de4381345d12: 2023-12-10 06:55:57,629 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=15, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=TestQuotaAdmin1 execute state=CREATE_TABLE_ADD_TO_META 2023-12-10 06:55:57,632 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"TestQuotaAdmin1,,1702191357579.0522cf92a3f0ad796f58de4381345d12.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1702191357631"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1702191357631"}]},"ts":"1702191357631"} 2023-12-10 06:55:57,634 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-12-10 06:55:57,636 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=15, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=TestQuotaAdmin1 execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-12-10 06:55:57,637 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestQuotaAdmin1","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1702191357636"}]},"ts":"1702191357636"} 2023-12-10 06:55:57,638 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=TestQuotaAdmin1, state=ENABLING in hbase:meta 2023-12-10 06:55:57,644 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-12-10 06:55:57,645 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-12-10 06:55:57,645 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-12-10 06:55:57,645 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-12-10 06:55:57,645 INFO [PEWorker-1] procedure2.ProcedureExecutor(1796): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestQuotaAdmin1, region=0522cf92a3f0ad796f58de4381345d12, ASSIGN}] 2023-12-10 06:55:57,647 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestQuotaAdmin1, region=0522cf92a3f0ad796f58de4381345d12, ASSIGN 2023-12-10 06:55:57,648 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=15, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestQuotaAdmin1, region=0522cf92a3f0ad796f58de4381345d12, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37997,1702191353702; forceNewPlan=false, retain=false 2023-12-10 06:55:57,798 INFO [jenkins-hbase4:40533] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-12-10 06:55:57,799 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=0522cf92a3f0ad796f58de4381345d12, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 06:55:57,799 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestQuotaAdmin1,,1702191357579.0522cf92a3f0ad796f58de4381345d12.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1702191357799"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1702191357799"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1702191357799"}]},"ts":"1702191357799"} 2023-12-10 06:55:57,802 INFO [PEWorker-3] procedure2.ProcedureExecutor(1796): Initialized subprocedures=[{pid=17, ppid=16, state=RUNNABLE; OpenRegionProcedure 0522cf92a3f0ad796f58de4381345d12, server=jenkins-hbase4.apache.org,37997,1702191353702}] 2023-12-10 06:55:57,837 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-12-10 06:55:57,955 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 06:55:57,959 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestQuotaAdmin1,,1702191357579.0522cf92a3f0ad796f58de4381345d12. 2023-12-10 06:55:57,960 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 0522cf92a3f0ad796f58de4381345d12, NAME => 'TestQuotaAdmin1,,1702191357579.0522cf92a3f0ad796f58de4381345d12.', STARTKEY => '', ENDKEY => ''} 2023-12-10 06:55:57,960 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestQuotaAdmin1 0522cf92a3f0ad796f58de4381345d12 2023-12-10 06:55:57,960 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestQuotaAdmin1,,1702191357579.0522cf92a3f0ad796f58de4381345d12.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-12-10 06:55:57,960 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 0522cf92a3f0ad796f58de4381345d12 2023-12-10 06:55:57,960 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 0522cf92a3f0ad796f58de4381345d12 2023-12-10 06:55:57,962 INFO [StoreOpener-0522cf92a3f0ad796f58de4381345d12-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf of region 0522cf92a3f0ad796f58de4381345d12 2023-12-10 06:55:57,964 DEBUG [StoreOpener-0522cf92a3f0ad796f58de4381345d12-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin1/0522cf92a3f0ad796f58de4381345d12/cf 2023-12-10 06:55:57,964 DEBUG [StoreOpener-0522cf92a3f0ad796f58de4381345d12-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin1/0522cf92a3f0ad796f58de4381345d12/cf 2023-12-10 06:55:57,964 INFO [StoreOpener-0522cf92a3f0ad796f58de4381345d12-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:10, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0522cf92a3f0ad796f58de4381345d12 columnFamilyName cf 2023-12-10 06:55:57,965 INFO [StoreOpener-0522cf92a3f0ad796f58de4381345d12-1] regionserver.HStore(310): Store=0522cf92a3f0ad796f58de4381345d12/cf, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-12-10 06:55:57,967 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin1/0522cf92a3f0ad796f58de4381345d12 2023-12-10 06:55:57,968 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin1/0522cf92a3f0ad796f58de4381345d12 2023-12-10 06:55:57,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 0522cf92a3f0ad796f58de4381345d12 2023-12-10 06:55:57,980 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin1/0522cf92a3f0ad796f58de4381345d12/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-12-10 06:55:57,981 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 0522cf92a3f0ad796f58de4381345d12; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73277908, jitterRate=0.09192591905593872}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-12-10 06:55:57,981 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 0522cf92a3f0ad796f58de4381345d12: 2023-12-10 06:55:57,982 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for TestQuotaAdmin1,,1702191357579.0522cf92a3f0ad796f58de4381345d12., pid=17, masterSystemTime=1702191357955 2023-12-10 06:55:57,984 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for TestQuotaAdmin1,,1702191357579.0522cf92a3f0ad796f58de4381345d12. 2023-12-10 06:55:57,984 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestQuotaAdmin1,,1702191357579.0522cf92a3f0ad796f58de4381345d12. 2023-12-10 06:55:57,985 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=0522cf92a3f0ad796f58de4381345d12, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 06:55:57,985 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestQuotaAdmin1,,1702191357579.0522cf92a3f0ad796f58de4381345d12.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1702191357985"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1702191357985"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1702191357985"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1702191357985"}]},"ts":"1702191357985"} 2023-12-10 06:55:57,990 INFO [PEWorker-5] procedure2.ProcedureExecutor(1953): Finished subprocedure pid=17, resume processing ppid=16 2023-12-10 06:55:57,990 INFO [PEWorker-5] procedure2.ProcedureExecutor(1457): Finished pid=17, ppid=16, state=SUCCESS; OpenRegionProcedure 0522cf92a3f0ad796f58de4381345d12, server=jenkins-hbase4.apache.org,37997,1702191353702 in 185 msec 2023-12-10 06:55:57,993 INFO [PEWorker-1] procedure2.ProcedureExecutor(1953): Finished subprocedure pid=16, resume processing ppid=15 2023-12-10 06:55:57,994 INFO [PEWorker-1] procedure2.ProcedureExecutor(1457): Finished pid=16, ppid=15, state=SUCCESS; TransitRegionStateProcedure table=TestQuotaAdmin1, region=0522cf92a3f0ad796f58de4381345d12, ASSIGN in 345 msec 2023-12-10 06:55:57,995 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=15, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=TestQuotaAdmin1 execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-12-10 06:55:57,995 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestQuotaAdmin1","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1702191357995"}]},"ts":"1702191357995"} 2023-12-10 06:55:57,997 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=TestQuotaAdmin1, state=ENABLED in hbase:meta 2023-12-10 06:55:58,000 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=15, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=TestQuotaAdmin1 execute state=CREATE_TABLE_POST_OPERATION 2023-12-10 06:55:58,002 INFO [PEWorker-2] procedure2.ProcedureExecutor(1457): Finished pid=15, state=SUCCESS; CreateTableProcedure table=TestQuotaAdmin1 in 420 msec 2023-12-10 06:55:58,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.MasterRpcServices(1230): Checking to see if procedure is done pid=15 2023-12-10 06:55:58,339 INFO [Listener at localhost/38369] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:TestQuotaAdmin1, procId: 15 completed 2023-12-10 06:55:58,339 DEBUG [Listener at localhost/38369] hbase.HBaseTestingUtility(3430): Waiting until all regions of table TestQuotaAdmin1 get assigned. Timeout = 60000ms 2023-12-10 06:55:58,340 INFO [Listener at localhost/38369] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-12-10 06:55:58,344 INFO [Listener at localhost/38369] hbase.HBaseTestingUtility(3484): All regions for table TestQuotaAdmin1 assigned to meta. Checking AM states. 2023-12-10 06:55:58,344 INFO [Listener at localhost/38369] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-12-10 06:55:58,344 INFO [Listener at localhost/38369] hbase.HBaseTestingUtility(3504): All regions for table TestQuotaAdmin1 assigned. 2023-12-10 06:55:58,345 INFO [Listener at localhost/38369] hbase.Waiter(180): Waiting up to [30,000] milli-secs(wait.for.ratio=[1]) 2023-12-10 06:55:58,356 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.HMaster$4(2118): Client=jenkins//172.31.14.131 create 'TestQuotaAdmin2', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '1'}}, {NAME => 'cf', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-12-10 06:55:58,357 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] procedure2.ProcedureExecutor(1075): Stored pid=18, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=TestQuotaAdmin2 2023-12-10 06:55:58,360 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=18, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=TestQuotaAdmin2 execute state=CREATE_TABLE_PRE_OPERATION 2023-12-10 06:55:58,360 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "TestQuotaAdmin2" procId is: 18 2023-12-10 06:55:58,361 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=18, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=TestQuotaAdmin2 execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-12-10 06:55:58,361 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.MasterRpcServices(1230): Checking to see if procedure is done pid=18 2023-12-10 06:55:58,363 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/default/TestQuotaAdmin2/700b1850706b553c6f995417c6bb917e 2023-12-10 06:55:58,364 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/default/TestQuotaAdmin2/700b1850706b553c6f995417c6bb917e empty. 2023-12-10 06:55:58,364 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/default/TestQuotaAdmin2/700b1850706b553c6f995417c6bb917e 2023-12-10 06:55:58,364 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived TestQuotaAdmin2 regions 2023-12-10 06:55:58,383 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/default/TestQuotaAdmin2/.tabledesc/.tableinfo.0000000001 2023-12-10 06:55:58,385 INFO [RegionOpenAndInit-TestQuotaAdmin2-pool-0] regionserver.HRegion(7675): creating {ENCODED => 700b1850706b553c6f995417c6bb917e, NAME => 'TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='TestQuotaAdmin2', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '1'}}, {NAME => 'cf', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp 2023-12-10 06:55:58,398 DEBUG [RegionOpenAndInit-TestQuotaAdmin2-pool-0] regionserver.HRegion(866): Instantiated TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-12-10 06:55:58,398 DEBUG [RegionOpenAndInit-TestQuotaAdmin2-pool-0] regionserver.HRegion(1604): Closing 700b1850706b553c6f995417c6bb917e, disabling compactions & flushes 2023-12-10 06:55:58,398 INFO [RegionOpenAndInit-TestQuotaAdmin2-pool-0] regionserver.HRegion(1626): Closing region TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e. 2023-12-10 06:55:58,398 DEBUG [RegionOpenAndInit-TestQuotaAdmin2-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e. 2023-12-10 06:55:58,398 DEBUG [RegionOpenAndInit-TestQuotaAdmin2-pool-0] regionserver.HRegion(1714): Acquired close lock on TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e. after waiting 0 ms 2023-12-10 06:55:58,398 DEBUG [RegionOpenAndInit-TestQuotaAdmin2-pool-0] regionserver.HRegion(1724): Updates disabled for region TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e. 2023-12-10 06:55:58,398 INFO [RegionOpenAndInit-TestQuotaAdmin2-pool-0] regionserver.HRegion(1838): Closed TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e. 2023-12-10 06:55:58,398 DEBUG [RegionOpenAndInit-TestQuotaAdmin2-pool-0] regionserver.HRegion(1558): Region close journal for 700b1850706b553c6f995417c6bb917e: 2023-12-10 06:55:58,402 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=18, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=TestQuotaAdmin2 execute state=CREATE_TABLE_ADD_TO_META 2023-12-10 06:55:58,404 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1702191358403"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1702191358403"}]},"ts":"1702191358403"} 2023-12-10 06:55:58,406 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-12-10 06:55:58,407 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=18, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=TestQuotaAdmin2 execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-12-10 06:55:58,407 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestQuotaAdmin2","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1702191358407"}]},"ts":"1702191358407"} 2023-12-10 06:55:58,409 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=TestQuotaAdmin2, state=ENABLING in hbase:meta 2023-12-10 06:55:58,413 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-12-10 06:55:58,414 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-12-10 06:55:58,414 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-12-10 06:55:58,414 DEBUG [PEWorker-3] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-12-10 06:55:58,414 INFO [PEWorker-3] procedure2.ProcedureExecutor(1796): Initialized subprocedures=[{pid=19, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestQuotaAdmin2, region=700b1850706b553c6f995417c6bb917e, ASSIGN}] 2023-12-10 06:55:58,416 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=19, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestQuotaAdmin2, region=700b1850706b553c6f995417c6bb917e, ASSIGN 2023-12-10 06:55:58,417 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=19, ppid=18, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestQuotaAdmin2, region=700b1850706b553c6f995417c6bb917e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37997,1702191353702; forceNewPlan=false, retain=false 2023-12-10 06:55:58,567 INFO [jenkins-hbase4:40533] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-12-10 06:55:58,568 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=19 updating hbase:meta row=700b1850706b553c6f995417c6bb917e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 06:55:58,568 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1702191358568"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1702191358568"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1702191358568"}]},"ts":"1702191358568"} 2023-12-10 06:55:58,571 INFO [PEWorker-5] procedure2.ProcedureExecutor(1796): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE; OpenRegionProcedure 700b1850706b553c6f995417c6bb917e, server=jenkins-hbase4.apache.org,37997,1702191353702}] 2023-12-10 06:55:58,613 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.MasterRpcServices(1230): Checking to see if procedure is done pid=18 2023-12-10 06:55:58,723 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 06:55:58,728 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e. 2023-12-10 06:55:58,728 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 700b1850706b553c6f995417c6bb917e, NAME => 'TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e.', STARTKEY => '', ENDKEY => ''} 2023-12-10 06:55:58,728 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestQuotaAdmin2 700b1850706b553c6f995417c6bb917e 2023-12-10 06:55:58,728 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-12-10 06:55:58,728 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 700b1850706b553c6f995417c6bb917e 2023-12-10 06:55:58,728 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 700b1850706b553c6f995417c6bb917e 2023-12-10 06:55:58,730 INFO [StoreOpener-700b1850706b553c6f995417c6bb917e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf of region 700b1850706b553c6f995417c6bb917e 2023-12-10 06:55:58,732 DEBUG [StoreOpener-700b1850706b553c6f995417c6bb917e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin2/700b1850706b553c6f995417c6bb917e/cf 2023-12-10 06:55:58,732 DEBUG [StoreOpener-700b1850706b553c6f995417c6bb917e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin2/700b1850706b553c6f995417c6bb917e/cf 2023-12-10 06:55:58,733 INFO [StoreOpener-700b1850706b553c6f995417c6bb917e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:10, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 700b1850706b553c6f995417c6bb917e columnFamilyName cf 2023-12-10 06:55:58,733 INFO [StoreOpener-700b1850706b553c6f995417c6bb917e-1] regionserver.HStore(310): Store=700b1850706b553c6f995417c6bb917e/cf, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-12-10 06:55:58,735 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin2/700b1850706b553c6f995417c6bb917e 2023-12-10 06:55:58,736 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin2/700b1850706b553c6f995417c6bb917e 2023-12-10 06:55:58,741 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 700b1850706b553c6f995417c6bb917e 2023-12-10 06:55:58,743 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin2/700b1850706b553c6f995417c6bb917e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-12-10 06:55:58,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 700b1850706b553c6f995417c6bb917e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61732989, jitterRate=-0.08010677993297577}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-12-10 06:55:58,744 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 700b1850706b553c6f995417c6bb917e: 2023-12-10 06:55:58,745 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e., pid=20, masterSystemTime=1702191358723 2023-12-10 06:55:58,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e. 2023-12-10 06:55:58,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e. 2023-12-10 06:55:58,748 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=19 updating hbase:meta row=700b1850706b553c6f995417c6bb917e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 06:55:58,748 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1702191358748"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1702191358748"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1702191358748"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1702191358748"}]},"ts":"1702191358748"} 2023-12-10 06:55:58,753 INFO [PEWorker-2] procedure2.ProcedureExecutor(1953): Finished subprocedure pid=20, resume processing ppid=19 2023-12-10 06:55:58,753 INFO [PEWorker-2] procedure2.ProcedureExecutor(1457): Finished pid=20, ppid=19, state=SUCCESS; OpenRegionProcedure 700b1850706b553c6f995417c6bb917e, server=jenkins-hbase4.apache.org,37997,1702191353702 in 179 msec 2023-12-10 06:55:58,756 INFO [PEWorker-3] procedure2.ProcedureExecutor(1953): Finished subprocedure pid=19, resume processing ppid=18 2023-12-10 06:55:58,757 INFO [PEWorker-3] procedure2.ProcedureExecutor(1457): Finished pid=19, ppid=18, state=SUCCESS; TransitRegionStateProcedure table=TestQuotaAdmin2, region=700b1850706b553c6f995417c6bb917e, ASSIGN in 339 msec 2023-12-10 06:55:58,758 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=18, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=TestQuotaAdmin2 execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-12-10 06:55:58,758 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestQuotaAdmin2","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1702191358758"}]},"ts":"1702191358758"} 2023-12-10 06:55:58,760 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=TestQuotaAdmin2, state=ENABLED in hbase:meta 2023-12-10 06:55:58,765 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=18, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=TestQuotaAdmin2 execute state=CREATE_TABLE_POST_OPERATION 2023-12-10 06:55:58,767 INFO [PEWorker-4] procedure2.ProcedureExecutor(1457): Finished pid=18, state=SUCCESS; CreateTableProcedure table=TestQuotaAdmin2 in 409 msec 2023-12-10 06:55:59,114 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.MasterRpcServices(1230): Checking to see if procedure is done pid=18 2023-12-10 06:55:59,114 INFO [Listener at localhost/38369] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:TestQuotaAdmin2, procId: 18 completed 2023-12-10 06:55:59,115 DEBUG [Listener at localhost/38369] hbase.HBaseTestingUtility(3430): Waiting until all regions of table TestQuotaAdmin2 get assigned. Timeout = 60000ms 2023-12-10 06:55:59,115 INFO [Listener at localhost/38369] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-12-10 06:55:59,118 INFO [Listener at localhost/38369] hbase.HBaseTestingUtility(3484): All regions for table TestQuotaAdmin2 assigned to meta. Checking AM states. 2023-12-10 06:55:59,118 INFO [Listener at localhost/38369] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-12-10 06:55:59,118 INFO [Listener at localhost/38369] hbase.HBaseTestingUtility(3504): All regions for table TestQuotaAdmin2 assigned. 2023-12-10 06:55:59,118 INFO [Listener at localhost/38369] hbase.Waiter(180): Waiting up to [30,000] milli-secs(wait.for.ratio=[1]) 2023-12-10 06:55:59,132 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.HMaster$15(3020): Client=jenkins//172.31.14.131 creating {NAME => 'TestNs'} 2023-12-10 06:55:59,134 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] procedure2.ProcedureExecutor(1075): Stored pid=21, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=TestNs 2023-12-10 06:55:59,140 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.MasterRpcServices(1230): Checking to see if procedure is done pid=21 2023-12-10 06:55:59,151 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-12-10 06:55:59,157 INFO [PEWorker-5] procedure2.ProcedureExecutor(1457): Finished pid=21, state=SUCCESS; CreateNamespaceProcedure, namespace=TestNs in 22 msec 2023-12-10 06:55:59,391 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.MasterRpcServices(1230): Checking to see if procedure is done pid=21 2023-12-10 06:55:59,393 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.HMaster$4(2118): Client=jenkins//172.31.14.131 create 'TestNs:TestTable', {NAME => 'cf', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-12-10 06:55:59,395 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] procedure2.ProcedureExecutor(1075): Stored pid=22, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=TestNs:TestTable 2023-12-10 06:55:59,398 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=22, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=TestNs:TestTable execute state=CREATE_TABLE_PRE_OPERATION 2023-12-10 06:55:59,398 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "TestNs" qualifier: "TestTable" procId is: 22 2023-12-10 06:55:59,399 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=22, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=TestNs:TestTable execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-12-10 06:55:59,399 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.MasterRpcServices(1230): Checking to see if procedure is done pid=22 2023-12-10 06:55:59,402 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/TestNs/TestTable/6f85f7a404e69950adc2db5f74a3bc3b 2023-12-10 06:55:59,402 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/TestNs/TestTable/79758411fb41135ec4572c165c4e2dc5 2023-12-10 06:55:59,403 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/TestNs/TestTable/6f85f7a404e69950adc2db5f74a3bc3b empty. 2023-12-10 06:55:59,403 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/TestNs/TestTable/79758411fb41135ec4572c165c4e2dc5 empty. 2023-12-10 06:55:59,403 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/TestNs/TestTable/6f85f7a404e69950adc2db5f74a3bc3b 2023-12-10 06:55:59,403 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/TestNs/TestTable/79758411fb41135ec4572c165c4e2dc5 2023-12-10 06:55:59,403 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived TestNs:TestTable regions 2023-12-10 06:55:59,419 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/TestNs/TestTable/.tabledesc/.tableinfo.0000000001 2023-12-10 06:55:59,420 INFO [RegionOpenAndInit-TestNs:TestTable-pool-0] regionserver.HRegion(7675): creating {ENCODED => 6f85f7a404e69950adc2db5f74a3bc3b, NAME => 'TestNs:TestTable,,1702191359393.6f85f7a404e69950adc2db5f74a3bc3b.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='TestNs:TestTable', {NAME => 'cf', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp 2023-12-10 06:55:59,421 INFO [RegionOpenAndInit-TestNs:TestTable-pool-1] regionserver.HRegion(7675): creating {ENCODED => 79758411fb41135ec4572c165c4e2dc5, NAME => 'TestNs:TestTable,1,1702191359393.79758411fb41135ec4572c165c4e2dc5.', STARTKEY => '1', ENDKEY => ''}, tableDescriptor='TestNs:TestTable', {NAME => 'cf', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp 2023-12-10 06:55:59,439 DEBUG [RegionOpenAndInit-TestNs:TestTable-pool-1] regionserver.HRegion(866): Instantiated TestNs:TestTable,1,1702191359393.79758411fb41135ec4572c165c4e2dc5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-12-10 06:55:59,439 DEBUG [RegionOpenAndInit-TestNs:TestTable-pool-1] regionserver.HRegion(1604): Closing 79758411fb41135ec4572c165c4e2dc5, disabling compactions & flushes 2023-12-10 06:55:59,439 INFO [RegionOpenAndInit-TestNs:TestTable-pool-1] regionserver.HRegion(1626): Closing region TestNs:TestTable,1,1702191359393.79758411fb41135ec4572c165c4e2dc5. 2023-12-10 06:55:59,439 DEBUG [RegionOpenAndInit-TestNs:TestTable-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on TestNs:TestTable,1,1702191359393.79758411fb41135ec4572c165c4e2dc5. 2023-12-10 06:55:59,439 DEBUG [RegionOpenAndInit-TestNs:TestTable-pool-1] regionserver.HRegion(1714): Acquired close lock on TestNs:TestTable,1,1702191359393.79758411fb41135ec4572c165c4e2dc5. after waiting 0 ms 2023-12-10 06:55:59,439 DEBUG [RegionOpenAndInit-TestNs:TestTable-pool-1] regionserver.HRegion(1724): Updates disabled for region TestNs:TestTable,1,1702191359393.79758411fb41135ec4572c165c4e2dc5. 2023-12-10 06:55:59,439 INFO [RegionOpenAndInit-TestNs:TestTable-pool-1] regionserver.HRegion(1838): Closed TestNs:TestTable,1,1702191359393.79758411fb41135ec4572c165c4e2dc5. 2023-12-10 06:55:59,439 DEBUG [RegionOpenAndInit-TestNs:TestTable-pool-0] regionserver.HRegion(866): Instantiated TestNs:TestTable,,1702191359393.6f85f7a404e69950adc2db5f74a3bc3b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-12-10 06:55:59,439 DEBUG [RegionOpenAndInit-TestNs:TestTable-pool-0] regionserver.HRegion(1604): Closing 6f85f7a404e69950adc2db5f74a3bc3b, disabling compactions & flushes 2023-12-10 06:55:59,439 INFO [RegionOpenAndInit-TestNs:TestTable-pool-0] regionserver.HRegion(1626): Closing region TestNs:TestTable,,1702191359393.6f85f7a404e69950adc2db5f74a3bc3b. 2023-12-10 06:55:59,439 DEBUG [RegionOpenAndInit-TestNs:TestTable-pool-1] regionserver.HRegion(1558): Region close journal for 79758411fb41135ec4572c165c4e2dc5: 2023-12-10 06:55:59,440 DEBUG [RegionOpenAndInit-TestNs:TestTable-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestNs:TestTable,,1702191359393.6f85f7a404e69950adc2db5f74a3bc3b. 2023-12-10 06:55:59,440 DEBUG [RegionOpenAndInit-TestNs:TestTable-pool-0] regionserver.HRegion(1714): Acquired close lock on TestNs:TestTable,,1702191359393.6f85f7a404e69950adc2db5f74a3bc3b. after waiting 0 ms 2023-12-10 06:55:59,440 DEBUG [RegionOpenAndInit-TestNs:TestTable-pool-0] regionserver.HRegion(1724): Updates disabled for region TestNs:TestTable,,1702191359393.6f85f7a404e69950adc2db5f74a3bc3b. 2023-12-10 06:55:59,440 INFO [RegionOpenAndInit-TestNs:TestTable-pool-0] regionserver.HRegion(1838): Closed TestNs:TestTable,,1702191359393.6f85f7a404e69950adc2db5f74a3bc3b. 2023-12-10 06:55:59,440 DEBUG [RegionOpenAndInit-TestNs:TestTable-pool-0] regionserver.HRegion(1558): Region close journal for 6f85f7a404e69950adc2db5f74a3bc3b: 2023-12-10 06:55:59,444 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=22, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=TestNs:TestTable execute state=CREATE_TABLE_ADD_TO_META 2023-12-10 06:55:59,445 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"TestNs:TestTable,1,1702191359393.79758411fb41135ec4572c165c4e2dc5.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1702191359445"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1702191359445"}]},"ts":"1702191359445"} 2023-12-10 06:55:59,445 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"TestNs:TestTable,,1702191359393.6f85f7a404e69950adc2db5f74a3bc3b.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1702191359445"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1702191359445"}]},"ts":"1702191359445"} 2023-12-10 06:55:59,451 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 2 regions to meta. 2023-12-10 06:55:59,452 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=22, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=TestNs:TestTable execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-12-10 06:55:59,452 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestNs:TestTable","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1702191359452"}]},"ts":"1702191359452"} 2023-12-10 06:55:59,454 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=TestNs:TestTable, state=ENABLING in hbase:meta 2023-12-10 06:55:59,459 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-12-10 06:55:59,461 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-12-10 06:55:59,461 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-12-10 06:55:59,461 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-12-10 06:55:59,462 INFO [PEWorker-1] procedure2.ProcedureExecutor(1796): Initialized subprocedures=[{pid=23, ppid=22, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestNs:TestTable, region=6f85f7a404e69950adc2db5f74a3bc3b, ASSIGN}, {pid=24, ppid=22, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestNs:TestTable, region=79758411fb41135ec4572c165c4e2dc5, ASSIGN}] 2023-12-10 06:55:59,464 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=23, ppid=22, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestNs:TestTable, region=6f85f7a404e69950adc2db5f74a3bc3b, ASSIGN 2023-12-10 06:55:59,465 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=22, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=TestNs:TestTable, region=79758411fb41135ec4572c165c4e2dc5, ASSIGN 2023-12-10 06:55:59,466 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=23, ppid=22, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestNs:TestTable, region=6f85f7a404e69950adc2db5f74a3bc3b, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,42073,1702191353745; forceNewPlan=false, retain=false 2023-12-10 06:55:59,466 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=22, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestNs:TestTable, region=79758411fb41135ec4572c165c4e2dc5, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37997,1702191353702; forceNewPlan=false, retain=false 2023-12-10 06:55:59,616 INFO [jenkins-hbase4:40533] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-12-10 06:55:59,617 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=79758411fb41135ec4572c165c4e2dc5, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 06:55:59,617 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=6f85f7a404e69950adc2db5f74a3bc3b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42073,1702191353745 2023-12-10 06:55:59,617 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestNs:TestTable,1,1702191359393.79758411fb41135ec4572c165c4e2dc5.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1702191359617"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1702191359617"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1702191359617"}]},"ts":"1702191359617"} 2023-12-10 06:55:59,617 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestNs:TestTable,,1702191359393.6f85f7a404e69950adc2db5f74a3bc3b.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1702191359617"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1702191359617"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1702191359617"}]},"ts":"1702191359617"} 2023-12-10 06:55:59,620 INFO [PEWorker-5] procedure2.ProcedureExecutor(1796): Initialized subprocedures=[{pid=25, ppid=24, state=RUNNABLE; OpenRegionProcedure 79758411fb41135ec4572c165c4e2dc5, server=jenkins-hbase4.apache.org,37997,1702191353702}] 2023-12-10 06:55:59,621 INFO [PEWorker-4] procedure2.ProcedureExecutor(1796): Initialized subprocedures=[{pid=26, ppid=23, state=RUNNABLE; OpenRegionProcedure 6f85f7a404e69950adc2db5f74a3bc3b, server=jenkins-hbase4.apache.org,42073,1702191353745}] 2023-12-10 06:55:59,650 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.MasterRpcServices(1230): Checking to see if procedure is done pid=22 2023-12-10 06:55:59,772 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 06:55:59,775 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,42073,1702191353745 2023-12-10 06:55:59,775 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(122): Using SIMPLE authentication for service=AdminService, sasl=false 2023-12-10 06:55:59,778 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43058, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-12-10 06:55:59,779 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestNs:TestTable,1,1702191359393.79758411fb41135ec4572c165c4e2dc5. 2023-12-10 06:55:59,779 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 79758411fb41135ec4572c165c4e2dc5, NAME => 'TestNs:TestTable,1,1702191359393.79758411fb41135ec4572c165c4e2dc5.', STARTKEY => '1', ENDKEY => ''} 2023-12-10 06:55:59,779 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestTable 79758411fb41135ec4572c165c4e2dc5 2023-12-10 06:55:59,780 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestNs:TestTable,1,1702191359393.79758411fb41135ec4572c165c4e2dc5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-12-10 06:55:59,780 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 79758411fb41135ec4572c165c4e2dc5 2023-12-10 06:55:59,780 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 79758411fb41135ec4572c165c4e2dc5 2023-12-10 06:55:59,782 INFO [StoreOpener-79758411fb41135ec4572c165c4e2dc5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf of region 79758411fb41135ec4572c165c4e2dc5 2023-12-10 06:55:59,784 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestNs:TestTable,,1702191359393.6f85f7a404e69950adc2db5f74a3bc3b. 2023-12-10 06:55:59,784 DEBUG [StoreOpener-79758411fb41135ec4572c165c4e2dc5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/TestNs/TestTable/79758411fb41135ec4572c165c4e2dc5/cf 2023-12-10 06:55:59,784 DEBUG [StoreOpener-79758411fb41135ec4572c165c4e2dc5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/TestNs/TestTable/79758411fb41135ec4572c165c4e2dc5/cf 2023-12-10 06:55:59,784 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6f85f7a404e69950adc2db5f74a3bc3b, NAME => 'TestNs:TestTable,,1702191359393.6f85f7a404e69950adc2db5f74a3bc3b.', STARTKEY => '', ENDKEY => '1'} 2023-12-10 06:55:59,785 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestTable 6f85f7a404e69950adc2db5f74a3bc3b 2023-12-10 06:55:59,785 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestNs:TestTable,,1702191359393.6f85f7a404e69950adc2db5f74a3bc3b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-12-10 06:55:59,785 INFO [StoreOpener-79758411fb41135ec4572c165c4e2dc5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:10, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 79758411fb41135ec4572c165c4e2dc5 columnFamilyName cf 2023-12-10 06:55:59,785 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6f85f7a404e69950adc2db5f74a3bc3b 2023-12-10 06:55:59,785 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6f85f7a404e69950adc2db5f74a3bc3b 2023-12-10 06:55:59,785 INFO [StoreOpener-79758411fb41135ec4572c165c4e2dc5-1] regionserver.HStore(310): Store=79758411fb41135ec4572c165c4e2dc5/cf, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-12-10 06:55:59,787 INFO [StoreOpener-6f85f7a404e69950adc2db5f74a3bc3b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf of region 6f85f7a404e69950adc2db5f74a3bc3b 2023-12-10 06:55:59,788 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/TestNs/TestTable/79758411fb41135ec4572c165c4e2dc5 2023-12-10 06:55:59,788 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/TestNs/TestTable/79758411fb41135ec4572c165c4e2dc5 2023-12-10 06:55:59,789 DEBUG [StoreOpener-6f85f7a404e69950adc2db5f74a3bc3b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/TestNs/TestTable/6f85f7a404e69950adc2db5f74a3bc3b/cf 2023-12-10 06:55:59,789 DEBUG [StoreOpener-6f85f7a404e69950adc2db5f74a3bc3b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/TestNs/TestTable/6f85f7a404e69950adc2db5f74a3bc3b/cf 2023-12-10 06:55:59,790 INFO [StoreOpener-6f85f7a404e69950adc2db5f74a3bc3b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:10, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6f85f7a404e69950adc2db5f74a3bc3b columnFamilyName cf 2023-12-10 06:55:59,790 INFO [StoreOpener-6f85f7a404e69950adc2db5f74a3bc3b-1] regionserver.HStore(310): Store=6f85f7a404e69950adc2db5f74a3bc3b/cf, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-12-10 06:55:59,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/TestNs/TestTable/6f85f7a404e69950adc2db5f74a3bc3b 2023-12-10 06:55:59,793 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/TestNs/TestTable/6f85f7a404e69950adc2db5f74a3bc3b 2023-12-10 06:55:59,793 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 79758411fb41135ec4572c165c4e2dc5 2023-12-10 06:55:59,797 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/TestNs/TestTable/79758411fb41135ec4572c165c4e2dc5/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-12-10 06:55:59,797 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6f85f7a404e69950adc2db5f74a3bc3b 2023-12-10 06:55:59,797 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 79758411fb41135ec4572c165c4e2dc5; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63337351, jitterRate=-0.056199923157691956}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-12-10 06:55:59,797 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 79758411fb41135ec4572c165c4e2dc5: 2023-12-10 06:55:59,798 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for TestNs:TestTable,1,1702191359393.79758411fb41135ec4572c165c4e2dc5., pid=25, masterSystemTime=1702191359772 2023-12-10 06:55:59,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/TestNs/TestTable/6f85f7a404e69950adc2db5f74a3bc3b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-12-10 06:55:59,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for TestNs:TestTable,1,1702191359393.79758411fb41135ec4572c165c4e2dc5. 2023-12-10 06:55:59,801 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestNs:TestTable,1,1702191359393.79758411fb41135ec4572c165c4e2dc5. 2023-12-10 06:55:59,802 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6f85f7a404e69950adc2db5f74a3bc3b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64841506, jitterRate=-0.033786267042160034}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-12-10 06:55:59,802 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6f85f7a404e69950adc2db5f74a3bc3b: 2023-12-10 06:55:59,802 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=79758411fb41135ec4572c165c4e2dc5, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 06:55:59,803 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for TestNs:TestTable,,1702191359393.6f85f7a404e69950adc2db5f74a3bc3b., pid=26, masterSystemTime=1702191359775 2023-12-10 06:55:59,803 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestNs:TestTable,1,1702191359393.79758411fb41135ec4572c165c4e2dc5.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1702191359802"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1702191359802"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1702191359802"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1702191359802"}]},"ts":"1702191359802"} 2023-12-10 06:55:59,809 INFO [PEWorker-2] procedure2.ProcedureExecutor(1953): Finished subprocedure pid=25, resume processing ppid=24 2023-12-10 06:55:59,809 INFO [PEWorker-2] procedure2.ProcedureExecutor(1457): Finished pid=25, ppid=24, state=SUCCESS; OpenRegionProcedure 79758411fb41135ec4572c165c4e2dc5, server=jenkins-hbase4.apache.org,37997,1702191353702 in 186 msec 2023-12-10 06:55:59,811 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for TestNs:TestTable,,1702191359393.6f85f7a404e69950adc2db5f74a3bc3b. 2023-12-10 06:55:59,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestNs:TestTable,,1702191359393.6f85f7a404e69950adc2db5f74a3bc3b. 2023-12-10 06:55:59,812 INFO [PEWorker-3] procedure2.ProcedureExecutor(1457): Finished pid=24, ppid=22, state=SUCCESS; TransitRegionStateProcedure table=TestNs:TestTable, region=79758411fb41135ec4572c165c4e2dc5, ASSIGN in 348 msec 2023-12-10 06:55:59,812 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=23 updating hbase:meta row=6f85f7a404e69950adc2db5f74a3bc3b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,42073,1702191353745 2023-12-10 06:55:59,813 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestNs:TestTable,,1702191359393.6f85f7a404e69950adc2db5f74a3bc3b.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1702191359812"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1702191359812"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1702191359812"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1702191359812"}]},"ts":"1702191359812"} 2023-12-10 06:55:59,817 INFO [PEWorker-4] procedure2.ProcedureExecutor(1953): Finished subprocedure pid=26, resume processing ppid=23 2023-12-10 06:55:59,818 INFO [PEWorker-4] procedure2.ProcedureExecutor(1457): Finished pid=26, ppid=23, state=SUCCESS; OpenRegionProcedure 6f85f7a404e69950adc2db5f74a3bc3b, server=jenkins-hbase4.apache.org,42073,1702191353745 in 193 msec 2023-12-10 06:55:59,819 INFO [PEWorker-5] procedure2.ProcedureExecutor(1953): Finished subprocedure pid=23, resume processing ppid=22 2023-12-10 06:55:59,820 INFO [PEWorker-5] procedure2.ProcedureExecutor(1457): Finished pid=23, ppid=22, state=SUCCESS; TransitRegionStateProcedure table=TestNs:TestTable, region=6f85f7a404e69950adc2db5f74a3bc3b, ASSIGN in 355 msec 2023-12-10 06:55:59,821 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=22, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=TestNs:TestTable execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-12-10 06:55:59,821 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestNs:TestTable","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1702191359821"}]},"ts":"1702191359821"} 2023-12-10 06:55:59,823 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=TestNs:TestTable, state=ENABLED in hbase:meta 2023-12-10 06:55:59,825 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=22, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=TestNs:TestTable execute state=CREATE_TABLE_POST_OPERATION 2023-12-10 06:55:59,827 INFO [PEWorker-1] procedure2.ProcedureExecutor(1457): Finished pid=22, state=SUCCESS; CreateTableProcedure table=TestNs:TestTable in 432 msec 2023-12-10 06:56:00,152 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.MasterRpcServices(1230): Checking to see if procedure is done pid=22 2023-12-10 06:56:00,152 INFO [Listener at localhost/38369] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: TestNs:TestTable, procId: 22 completed 2023-12-10 06:56:00,152 DEBUG [Listener at localhost/38369] hbase.HBaseTestingUtility(3430): Waiting until all regions of table TestNs:TestTable get assigned. Timeout = 60000ms 2023-12-10 06:56:00,153 INFO [Listener at localhost/38369] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-12-10 06:56:00,156 INFO [Listener at localhost/38369] hbase.HBaseTestingUtility(3484): All regions for table TestNs:TestTable assigned to meta. Checking AM states. 2023-12-10 06:56:00,156 INFO [Listener at localhost/38369] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-12-10 06:56:00,156 INFO [Listener at localhost/38369] hbase.HBaseTestingUtility(3504): All regions for table TestNs:TestTable assigned. 2023-12-10 06:56:00,157 INFO [Listener at localhost/38369] hbase.Waiter(180): Waiting up to [30,000] milli-secs(wait.for.ratio=[1]) 2023-12-10 06:56:00,165 DEBUG [hconnection-0x505566b3-shared-pool-1] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-12-10 06:56:00,167 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43070, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-12-10 06:56:00,170 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-12-10 06:56:00,172 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41560, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=MasterService 2023-12-10 06:56:00,184 DEBUG [hconnection-0x214824e3-metaLookup-shared--pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-12-10 06:56:00,195 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41696, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=ClientService 2023-12-10 06:56:00,206 INFO [Listener at localhost/38369] hbase.ResourceChecker(147): before: quotas.TestClusterScopeQuotaThrottle#testUserTableClusterScopeQuota Thread=306, OpenFileDescriptor=630, MaxFileDescriptor=60000, SystemLoadAverage=304, ProcessCount=171, AvailableMemoryMB=5741 2023-12-10 06:56:00,524 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(184): QuotaCache 2023-12-10 06:56:00,524 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(185): {default=QuotaState(ts=1702194960273 bypass), TestNs=QuotaState(ts=1702194960273 bypass)} 2023-12-10 06:56:00,524 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(186): {TestQuotaAdmin0=QuotaState(ts=1702194960273 bypass), TestNs:TestTable=QuotaState(ts=1702194960273 bypass), TestQuotaAdmin2=QuotaState(ts=1702194960273 bypass), TestQuotaAdmin1=QuotaState(ts=1702194960273 bypass)} 2023-12-10 06:56:00,524 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(187): {jenkins=UserQuotaState(ts=1702194960273 [ TestNs:TestTable ])} 2023-12-10 06:56:00,524 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(188): {all=QuotaState(ts=1702194960273 bypass)} 2023-12-10 06:56:00,775 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(184): QuotaCache 2023-12-10 06:56:00,775 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(185): {TestNs=QuotaState(ts=1702194960273 bypass)} 2023-12-10 06:56:00,775 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(186): {TestNs:TestTable=QuotaState(ts=1702194960273 bypass)} 2023-12-10 06:56:00,775 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(187): {jenkins=UserQuotaState(ts=1702194960273 [ TestNs:TestTable ])} 2023-12-10 06:56:00,775 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(188): {all=QuotaState(ts=1702194960273 bypass)} 2023-12-10 06:56:00,788 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestNs:TestTable numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 06:56:00,788 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] ipc.CallRunner(144): callId: 90 service: ClientService methodName: Get size: 116 connection: 172.31.14.131:41680 deadline: 1702194970273, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 06:56:00,855 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-12-10 06:56:00,914 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'TestNs:TestTable' 2023-12-10 06:56:00,915 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'TestQuotaAdmin0' 2023-12-10 06:56:00,916 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-12-10 06:56:00,916 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-12-10 06:56:00,916 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'TestQuotaAdmin2' 2023-12-10 06:56:00,917 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:quota' 2023-12-10 06:56:00,917 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'TestQuotaAdmin1' 2023-12-10 06:56:00,931 INFO [regionserver/jenkins-hbase4:0.Chore.1] regionserver.HRegionServer$PeriodicMemStoreFlusher(1921): MemstoreFlusherChore requesting flush of hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b. because dca2e6ecbeef49fb8929fd36dd74df8b/q has an old edit so flush to free WALs after random delay 280252 ms 2023-12-10 06:56:00,932 INFO [regionserver/jenkins-hbase4:0.Chore.1] regionserver.HRegionServer$PeriodicMemStoreFlusher(1921): MemstoreFlusherChore requesting flush of hbase:meta,,1.1588230740 because 1588230740/info has an old edit so flush to free WALs after random delay 11836 ms 2023-12-10 06:56:00,932 INFO [regionserver/jenkins-hbase4:0.Chore.1] regionserver.HRegionServer$PeriodicMemStoreFlusher(1921): MemstoreFlusherChore requesting flush of hbase:namespace,,1702191355536.3b6714111b5dc2b8d14208f4c54166e3. because 3b6714111b5dc2b8d14208f4c54166e3/info has an old edit so flush to free WALs after random delay 5717 ms 2023-12-10 06:56:01,057 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestNs:TestTable numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 06:56:01,057 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] ipc.CallRunner(144): callId: 92 service: ClientService methodName: Get size: 116 connection: 172.31.14.131:41680 deadline: 1702194970273, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 06:56:01,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestNs:TestTable numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 06:56:01,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] ipc.CallRunner(144): callId: 94 service: ClientService methodName: Get size: 116 connection: 172.31.14.131:41680 deadline: 1702194970273, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 06:56:02,319 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestNs:TestTable numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 06:56:02,319 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] ipc.CallRunner(144): callId: 96 service: ClientService methodName: Get size: 116 connection: 172.31.14.131:41680 deadline: 1702194970273, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 06:56:02,723 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: Master,sub=Coprocessor.Master.CP_org.apache.hadoop.hbase.quotas.MasterQuotasObserver 2023-12-10 06:56:02,724 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering Master,sub=Coprocessor.Master.CP_org.apache.hadoop.hbase.quotas.MasterQuotasObserver Metrics about HBase MasterObservers 2023-12-10 06:56:02,724 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-12-10 06:56:02,724 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-12-10 06:56:03,578 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestNs:TestTable numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 06:56:03,578 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] ipc.CallRunner(144): callId: 98 service: ClientService methodName: Get size: 116 connection: 172.31.14.131:41680 deadline: 1702194970273, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 06:56:06,103 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestNs:TestTable numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 06:56:06,103 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] ipc.CallRunner(144): callId: 100 service: ClientService methodName: Get size: 116 connection: 172.31.14.131:41680 deadline: 1702194970273, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 06:56:11,114 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestNs:TestTable numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 06:56:11,114 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] ipc.CallRunner(144): callId: 102 service: ClientService methodName: Get size: 116 connection: 172.31.14.131:41680 deadline: 1702194970273, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 06:56:11,116 DEBUG [Listener at localhost/38369] client.RpcRetryingCallerImpl(129): Call exception, tries=6, retries=16, started=0 ms ago, cancelled=false, msg=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) , details=row 'row-10' on table 'TestNs:TestTable' at region=TestNs:TestTable,1,1702191359393.79758411fb41135ec4572c165c4e2dc5., hostname=jenkins-hbase4.apache.org,37997,1702191353702, seqNum=2, see https://s.apache.org/timeout, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:276) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:261) at org.apache.hadoop.hbase.client.RegionServerCallable.call(RegionServerCallable.java:126) at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:104) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:370) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:343) at org.apache.hadoop.hbase.quotas.ThrottleQuotaTestUtil.doGets(ThrottleQuotaTestUtil.java:95) at org.apache.hadoop.hbase.quotas.TestClusterScopeQuotaThrottle.testUserTableClusterScopeQuota(TestClusterScopeQuotaThrottle.java:224) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.quotas.RpcThrottlingException): org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:381) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:87) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:415) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:411) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:193) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:214) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-12-10 06:56:11,116 ERROR [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(100): get failed after nRetries=10 java.net.SocketTimeoutException: callTimeout=10000, callDuration=10050: org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) row 'row-10' on table 'TestNs:TestTable' at region=TestNs:TestTable,1,1702191359393.79758411fb41135ec4572c165c4e2dc5., hostname=jenkins-hbase4.apache.org,37997,1702191353702, seqNum=2 at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:156) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:370) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:343) at org.apache.hadoop.hbase.quotas.ThrottleQuotaTestUtil.doGets(ThrottleQuotaTestUtil.java:95) at org.apache.hadoop.hbase.quotas.TestClusterScopeQuotaThrottle.testUserTableClusterScopeQuota(TestClusterScopeQuotaThrottle.java:224) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.quotas.RpcThrottlingException: org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:276) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:261) at org.apache.hadoop.hbase.client.RegionServerCallable.call(RegionServerCallable.java:126) at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:104) ... 29 more Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.quotas.RpcThrottlingException): org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:381) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:87) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:415) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:411) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:193) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:214) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-12-10 06:56:11,119 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestNs:TestTable numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 06:56:11,119 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] ipc.CallRunner(144): callId: 103 service: ClientService methodName: Get size: 115 connection: 172.31.14.131:41680 deadline: 1702194970273, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 06:56:11,373 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestNs:TestTable numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 06:56:11,373 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] ipc.CallRunner(144): callId: 105 service: ClientService methodName: Get size: 115 connection: 172.31.14.131:41680 deadline: 1702194970273, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 06:56:11,878 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestNs:TestTable numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 06:56:11,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] ipc.CallRunner(144): callId: 107 service: ClientService methodName: Get size: 115 connection: 172.31.14.131:41680 deadline: 1702194970273, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 06:56:12,632 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestNs:TestTable numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 06:56:12,632 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] ipc.CallRunner(144): callId: 109 service: ClientService methodName: Get size: 115 connection: 172.31.14.131:41680 deadline: 1702194970273, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 06:56:13,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestNs:TestTable numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 06:56:13,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] ipc.CallRunner(144): callId: 111 service: ClientService methodName: Get size: 115 connection: 172.31.14.131:41680 deadline: 1702194970273, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 06:56:16,400 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestNs:TestTable numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 06:56:16,400 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] ipc.CallRunner(144): callId: 113 service: ClientService methodName: Get size: 115 connection: 172.31.14.131:41680 deadline: 1702194970273, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 06:56:21,451 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestNs:TestTable numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 06:56:21,451 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] ipc.CallRunner(144): callId: 115 service: ClientService methodName: Get size: 115 connection: 172.31.14.131:41680 deadline: 1702194970273, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 06:56:21,452 DEBUG [Listener at localhost/38369] client.RpcRetryingCallerImpl(129): Call exception, tries=6, retries=16, started=0 ms ago, cancelled=false, msg=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) , details=row 'row-0' on table 'TestNs:TestTable' at region=TestNs:TestTable,1,1702191359393.79758411fb41135ec4572c165c4e2dc5., hostname=jenkins-hbase4.apache.org,37997,1702191353702, seqNum=2, see https://s.apache.org/timeout, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:276) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:261) at org.apache.hadoop.hbase.client.RegionServerCallable.call(RegionServerCallable.java:126) at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:104) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:370) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:343) at org.apache.hadoop.hbase.quotas.ThrottleQuotaTestUtil.doGets(ThrottleQuotaTestUtil.java:95) at org.apache.hadoop.hbase.quotas.TestClusterScopeQuotaThrottle.testUserTableClusterScopeQuota(TestClusterScopeQuotaThrottle.java:224) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.quotas.RpcThrottlingException): org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:381) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:87) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:415) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:411) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:193) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:214) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-12-10 06:56:21,453 ERROR [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(100): get failed after nRetries=0 java.net.SocketTimeoutException: callTimeout=10000, callDuration=10004: org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) row 'row-0' on table 'TestNs:TestTable' at region=TestNs:TestTable,1,1702191359393.79758411fb41135ec4572c165c4e2dc5., hostname=jenkins-hbase4.apache.org,37997,1702191353702, seqNum=2 at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:156) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:370) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:343) at org.apache.hadoop.hbase.quotas.ThrottleQuotaTestUtil.doGets(ThrottleQuotaTestUtil.java:95) at org.apache.hadoop.hbase.quotas.TestClusterScopeQuotaThrottle.testUserTableClusterScopeQuota(TestClusterScopeQuotaThrottle.java:224) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.quotas.RpcThrottlingException: org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:276) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:261) at org.apache.hadoop.hbase.client.RegionServerCallable.call(RegionServerCallable.java:126) at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:104) ... 29 more Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.quotas.RpcThrottlingException): org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:381) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:87) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:415) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:411) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:193) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:214) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-12-10 06:56:21,474 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 3b6714111b5dc2b8d14208f4c54166e3 1/1 column families, dataSize=117 B heapSize=600 B 2023-12-10 06:56:21,548 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=117 B at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/namespace/3b6714111b5dc2b8d14208f4c54166e3/.tmp/info/cd7fb452c32f4502b93787b0240bead2 2023-12-10 06:56:21,596 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/namespace/3b6714111b5dc2b8d14208f4c54166e3/.tmp/info/cd7fb452c32f4502b93787b0240bead2 as hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/namespace/3b6714111b5dc2b8d14208f4c54166e3/info/cd7fb452c32f4502b93787b0240bead2 2023-12-10 06:56:21,606 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/namespace/3b6714111b5dc2b8d14208f4c54166e3/info/cd7fb452c32f4502b93787b0240bead2, entries=3, sequenceid=7, filesize=4.9 K 2023-12-10 06:56:21,609 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~117 B/117, heapSize ~584 B/584, currentSize=0 B/0 for 3b6714111b5dc2b8d14208f4c54166e3 in 0ms, sequenceid=7, compaction requested=false 2023-12-10 06:56:21,610 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 3b6714111b5dc2b8d14208f4c54166e3: 2023-12-10 06:56:21,610 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2489): Flush column family: info of 1588230740 because time of oldest edit=1702191355661 is > 3600000 from now =1702198560273 2023-12-10 06:56:21,611 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2489): Flush column family: table of 1588230740 because time of oldest edit=1702191355676 is > 3600000 from now =1702198560273 2023-12-10 06:56:21,611 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 1588230740 2/3 column families, dataSize=8.78 KB heapSize=15.55 KB; table={dataSize=558 B, heapSize=1.66 KB, offHeapSize=0 B}; info={dataSize=8.24 KB, heapSize=13.65 KB, offHeapSize=0 B} 2023-12-10 06:56:21,627 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=8.24 KB at sequenceid=36 (bloomFilter=false), to=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740/.tmp/info/b96603ddfe764db4a6aa14b1369a4034 2023-12-10 06:56:21,650 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=558 B at sequenceid=36 (bloomFilter=false), to=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740/.tmp/table/7fd2fd772f6a4eb78d82e265d377f73a 2023-12-10 06:56:21,657 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740/.tmp/info/b96603ddfe764db4a6aa14b1369a4034 as hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740/info/b96603ddfe764db4a6aa14b1369a4034 2023-12-10 06:56:21,665 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740/info/b96603ddfe764db4a6aa14b1369a4034, entries=70, sequenceid=36, filesize=13.1 K 2023-12-10 06:56:21,666 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740/.tmp/table/7fd2fd772f6a4eb78d82e265d377f73a as hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740/table/7fd2fd772f6a4eb78d82e265d377f73a 2023-12-10 06:56:21,673 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740/table/7fd2fd772f6a4eb78d82e265d377f73a, entries=12, sequenceid=36, filesize=5.1 K 2023-12-10 06:56:21,674 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~8.78 KB/8992, heapSize ~15.27 KB/15640, currentSize=0 B/0 for 1588230740 in 0ms, sequenceid=36, compaction requested=false 2023-12-10 06:56:21,674 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 1588230740: 2023-12-10 06:56:21,675 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2489): Flush column family: q of dca2e6ecbeef49fb8929fd36dd74df8b because time of oldest edit=1702191360269 is > 3600000 from now =1702198560273 2023-12-10 06:56:21,675 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing dca2e6ecbeef49fb8929fd36dd74df8b 1/2 column families, dataSize=122 B heapSize=856 B; q={dataSize=92 B, heapSize=496 B, offHeapSize=0 B} 2023-12-10 06:56:21,691 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=92 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/.tmp/q/a0e8d98621be4825946bd6a3dee2f5e8 2023-12-10 06:56:21,698 INFO [MemStoreFlusher.0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a0e8d98621be4825946bd6a3dee2f5e8 2023-12-10 06:56:21,700 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/.tmp/q/a0e8d98621be4825946bd6a3dee2f5e8 as hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/q/a0e8d98621be4825946bd6a3dee2f5e8 2023-12-10 06:56:21,706 INFO [MemStoreFlusher.0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a0e8d98621be4825946bd6a3dee2f5e8 2023-12-10 06:56:21,706 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/q/a0e8d98621be4825946bd6a3dee2f5e8, entries=1, sequenceid=6, filesize=4.9 K 2023-12-10 06:56:21,707 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~92 B/92, heapSize ~480 B/480, currentSize=30 B/30 for dca2e6ecbeef49fb8929fd36dd74df8b in 0ms, sequenceid=6, compaction requested=false 2023-12-10 06:56:21,708 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for dca2e6ecbeef49fb8929fd36dd74df8b: 2023-12-10 06:56:21,710 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(184): QuotaCache 2023-12-10 06:56:21,710 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(185): {default=QuotaState(ts=1702198560273 bypass), TestNs=QuotaState(ts=1702198560273 bypass)} 2023-12-10 06:56:21,710 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(186): {TestQuotaAdmin0=QuotaState(ts=1702198560273 bypass), TestNs:TestTable=QuotaState(ts=1702198560273 bypass), TestQuotaAdmin2=QuotaState(ts=1702198560273 bypass), TestQuotaAdmin1=QuotaState(ts=1702198560273 bypass)} 2023-12-10 06:56:21,710 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(187): {jenkins=UserQuotaState(ts=1702198560273 bypass)} 2023-12-10 06:56:21,710 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(188): {all=QuotaState(ts=1702198560273 bypass)} 2023-12-10 06:56:21,938 INFO [regionserver/jenkins-hbase4:0.Chore.1] regionserver.HRegionServer$PeriodicMemStoreFlusher(1921): MemstoreFlusherChore requesting flush of hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b. because dca2e6ecbeef49fb8929fd36dd74df8b/u has an old edit so flush to free WALs after random delay 225172 ms 2023-12-10 06:56:21,960 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(184): QuotaCache 2023-12-10 06:56:21,960 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(185): {TestNs=QuotaState(ts=1702198560273 bypass)} 2023-12-10 06:56:21,961 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(186): {TestNs:TestTable=QuotaState(ts=1702198560273 bypass)} 2023-12-10 06:56:21,961 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(187): {jenkins=UserQuotaState(ts=1702198560273 bypass)} 2023-12-10 06:56:21,961 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(188): {all=QuotaState(ts=1702198560273 bypass)} 2023-12-10 06:56:21,975 INFO [Listener at localhost/38369] hbase.ResourceChecker(175): after: quotas.TestClusterScopeQuotaThrottle#testUserTableClusterScopeQuota Thread=298 (was 306), OpenFileDescriptor=627 (was 630), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=206 (was 304), ProcessCount=171 (was 171), AvailableMemoryMB=5678 (was 5741) 2023-12-10 06:56:21,986 INFO [Listener at localhost/38369] hbase.ResourceChecker(147): before: quotas.TestClusterScopeQuotaThrottle#testUserNamespaceClusterScopeQuota Thread=298, OpenFileDescriptor=627, MaxFileDescriptor=60000, SystemLoadAverage=206, ProcessCount=171, AvailableMemoryMB=5677 2023-12-10 06:56:22,009 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2489): Flush column family: u of dca2e6ecbeef49fb8929fd36dd74df8b because time of oldest edit=1702194960273 is > 3600000 from now =1702202160273 2023-12-10 06:56:22,009 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing dca2e6ecbeef49fb8929fd36dd74df8b 1/2 column families, dataSize=146 B heapSize=880 B; u={dataSize=30 B, heapSize=360 B, offHeapSize=0 B} 2023-12-10 06:56:22,025 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=30 B at sequenceid=11 (bloomFilter=true), to=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/.tmp/u/dd54a76871ef426db9514b247981de5a 2023-12-10 06:56:22,032 INFO [MemStoreFlusher.0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dd54a76871ef426db9514b247981de5a 2023-12-10 06:56:22,033 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/.tmp/u/dd54a76871ef426db9514b247981de5a as hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/u/dd54a76871ef426db9514b247981de5a 2023-12-10 06:56:22,040 INFO [MemStoreFlusher.0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dd54a76871ef426db9514b247981de5a 2023-12-10 06:56:22,040 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/u/dd54a76871ef426db9514b247981de5a, entries=1, sequenceid=11, filesize=4.9 K 2023-12-10 06:56:22,041 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~30 B/30, heapSize ~344 B/344, currentSize=116 B/116 for dca2e6ecbeef49fb8929fd36dd74df8b in 0ms, sequenceid=11, compaction requested=false 2023-12-10 06:56:22,041 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for dca2e6ecbeef49fb8929fd36dd74df8b: 2023-12-10 06:56:22,251 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(184): QuotaCache 2023-12-10 06:56:22,252 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(185): {default=QuotaState(ts=1702202181773 bypass), TestNs=QuotaState(ts=1702202181773 bypass)} 2023-12-10 06:56:22,253 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(186): {TestQuotaAdmin0=QuotaState(ts=1702202191673 bypass), TestNs:TestTable=QuotaState(ts=1702202191673 bypass), TestQuotaAdmin2=QuotaState(ts=1702202191673 bypass), TestQuotaAdmin1=QuotaState(ts=1702202191673 bypass)} 2023-12-10 06:56:22,253 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(187): {jenkins=UserQuotaState(ts=1702202203273 [ default ])} 2023-12-10 06:56:22,253 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(188): {all=QuotaState(ts=1702202210773 bypass)} 2023-12-10 06:56:22,511 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702202323073","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":12200,"client":"172.31.14.131:41696","queuetimems":600,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:22,570 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":2,"starttimems":"1702202996573","responsesize":"20","method":"Multi","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., for 2 action(s) and 1st row key=t.TestQuotaAdmin0","processingtimems":11000,"client":"172.31.14.131:41696","multi.service_calls":0,"queuetimems":800,"class":"MiniHBaseClusterRegionServer","multi.mutations":0} 2023-12-10 06:56:22,643 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":2,"starttimems":"1702203857573","responsesize":"20","method":"Multi","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., for 2 action(s) and 1st row key=t.TestQuotaAdmin0","processingtimems":10500,"client":"172.31.14.131:41696","multi.service_calls":0,"queuetimems":600,"class":"MiniHBaseClusterRegionServer","multi.mutations":0} 2023-12-10 06:56:22,644 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702203890873","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":11600,"client":"172.31.14.131:41696","queuetimems":500,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:22,657 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702204062373","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":13000,"client":"172.31.14.131:41696","queuetimems":700,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:22,669 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":2,"starttimems":"1702204212373","responsesize":"20","method":"Multi","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., for 2 action(s) and 1st row key=t.TestQuotaAdmin0","processingtimems":11500,"client":"172.31.14.131:41696","multi.service_calls":0,"queuetimems":900,"class":"MiniHBaseClusterRegionServer","multi.mutations":0} 2023-12-10 06:56:22,670 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702204252673","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":12600,"client":"172.31.14.131:41696","queuetimems":1500,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:22,683 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":2,"starttimems":"1702204420973","responsesize":"20","method":"Multi","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., for 2 action(s) and 1st row key=t.TestQuotaAdmin0","processingtimems":12700,"client":"172.31.14.131:41696","multi.service_calls":0,"queuetimems":600,"class":"MiniHBaseClusterRegionServer","multi.mutations":0} 2023-12-10 06:56:22,685 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702204459573","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":13700,"client":"172.31.14.131:41696","queuetimems":500,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:22,694 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] ipc.RpcServer(528): (responseTooSlow): {"call":"GetClusterStatus(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$GetClusterStatusRequest)","starttimems":"1702204561773","responsesize":"339","method":"GetClusterStatus","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$GetClusterStatusRequest","processingtimems":10700,"client":"172.31.14.131:41560","queuetimems":1200,"class":"HMaster"} 2023-12-10 06:56:22,698 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":2,"starttimems":"1702204639873","responsesize":"20","method":"Multi","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., for 2 action(s) and 1st row key=t.TestQuotaAdmin0","processingtimems":17800,"client":"172.31.14.131:41696","multi.service_calls":0,"queuetimems":1800,"class":"MiniHBaseClusterRegionServer","multi.mutations":0} 2023-12-10 06:56:22,699 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702204688573","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":16200,"client":"172.31.14.131:41696","queuetimems":800,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:22,710 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] ipc.RpcServer(528): (responseTooSlow): {"call":"GetClusterStatus(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$GetClusterStatusRequest)","starttimems":"1702204798073","responsesize":"339","method":"GetClusterStatus","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$GetClusterStatusRequest","processingtimems":14100,"client":"172.31.14.131:41560","queuetimems":1000,"class":"HMaster"} 2023-12-10 06:56:22,713 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":2,"starttimems":"1702204886973","responsesize":"20","method":"Multi","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., for 2 action(s) and 1st row key=t.TestQuotaAdmin0","processingtimems":13400,"client":"172.31.14.131:41696","multi.service_calls":0,"queuetimems":900,"class":"MiniHBaseClusterRegionServer","multi.mutations":0} 2023-12-10 06:56:22,714 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702204929673","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":15400,"client":"172.31.14.131:41696","queuetimems":1000,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:22,726 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":2,"starttimems":"1702205098573","responsesize":"20","method":"Multi","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., for 2 action(s) and 1st row key=t.TestQuotaAdmin0","processingtimems":10700,"client":"172.31.14.131:41696","multi.service_calls":0,"queuetimems":800,"class":"MiniHBaseClusterRegionServer","multi.mutations":0} 2023-12-10 06:56:22,727 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702205139173","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":12900,"client":"172.31.14.131:41696","queuetimems":600,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:22,734 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] ipc.RpcServer(528): (responseTooSlow): {"call":"GetClusterStatus(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$GetClusterStatusRequest)","starttimems":"1702205230273","responsesize":"339","method":"GetClusterStatus","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$GetClusterStatusRequest","processingtimems":10500,"client":"172.31.14.131:41560","queuetimems":900,"class":"HMaster"} 2023-12-10 06:56:22,739 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702205328173","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":10800,"client":"172.31.14.131:41696","queuetimems":1000,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:22,752 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702205541173","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":12500,"client":"172.31.14.131:41696","queuetimems":600,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:22,761 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] ipc.RpcServer(528): (responseTooSlow): {"call":"GetClusterStatus(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$GetClusterStatusRequest)","starttimems":"1702205634973","responsesize":"339","method":"GetClusterStatus","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$GetClusterStatusRequest","processingtimems":10300,"client":"172.31.14.131:41560","queuetimems":900,"class":"HMaster"} 2023-12-10 06:56:22,764 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":2,"starttimems":"1702205695473","responsesize":"20","method":"Multi","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., for 2 action(s) and 1st row key=t.TestQuotaAdmin0","processingtimems":10400,"client":"172.31.14.131:41696","multi.service_calls":0,"queuetimems":500,"class":"MiniHBaseClusterRegionServer","multi.mutations":0} 2023-12-10 06:56:22,765 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702205727573","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":12000,"client":"172.31.14.131:41696","queuetimems":700,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:22,773 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] ipc.RpcServer(528): (responseTooSlow): {"call":"GetClusterStatus(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$GetClusterStatusRequest)","starttimems":"1702205817873","responsesize":"339","method":"GetClusterStatus","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$GetClusterStatusRequest","processingtimems":12100,"client":"172.31.14.131:41560","queuetimems":700,"class":"HMaster"} 2023-12-10 06:56:22,777 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702205927973","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":16600,"client":"172.31.14.131:41696","queuetimems":700,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:22,795 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":2,"starttimems":"1702206152673","responsesize":"20","method":"Multi","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., for 2 action(s) and 1st row key=t.TestQuotaAdmin0","processingtimems":11900,"client":"172.31.14.131:41696","multi.service_calls":0,"queuetimems":500,"class":"MiniHBaseClusterRegionServer","multi.mutations":0} 2023-12-10 06:56:22,796 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702206195073","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":15800,"client":"172.31.14.131:41696","queuetimems":600,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:22,810 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":2,"starttimems":"1702206364973","responsesize":"20","method":"Multi","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., for 2 action(s) and 1st row key=t.TestQuotaAdmin0","processingtimems":10100,"client":"172.31.14.131:41696","multi.service_calls":0,"queuetimems":500,"class":"MiniHBaseClusterRegionServer","multi.mutations":0} 2023-12-10 06:56:22,811 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702206398473","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":16700,"client":"172.31.14.131:41696","queuetimems":400,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:22,846 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":2,"starttimems":"1702206909773","responsesize":"20","method":"Multi","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., for 2 action(s) and 1st row key=t.TestQuotaAdmin0","processingtimems":11500,"client":"172.31.14.131:41696","multi.service_calls":0,"queuetimems":700,"class":"MiniHBaseClusterRegionServer","multi.mutations":0} 2023-12-10 06:56:22,848 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702206954673","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":12700,"client":"172.31.14.131:41696","queuetimems":700,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:22,863 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":2,"starttimems":"1702207144973","responsesize":"20","method":"Multi","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., for 2 action(s) and 1st row key=t.TestQuotaAdmin0","processingtimems":13800,"client":"172.31.14.131:41696","multi.service_calls":0,"queuetimems":500,"class":"MiniHBaseClusterRegionServer","multi.mutations":0} 2023-12-10 06:56:22,875 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":2,"starttimems":"1702207342973","responsesize":"20","method":"Multi","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., for 2 action(s) and 1st row key=t.TestQuotaAdmin0","processingtimems":12800,"client":"172.31.14.131:41696","multi.service_calls":0,"queuetimems":500,"class":"MiniHBaseClusterRegionServer","multi.mutations":0} 2023-12-10 06:56:22,877 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702207383673","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":12900,"client":"172.31.14.131:41696","queuetimems":700,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:22,891 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":2,"starttimems":"1702207548073","responsesize":"20","method":"Multi","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., for 2 action(s) and 1st row key=t.TestQuotaAdmin0","processingtimems":13400,"client":"172.31.14.131:41696","multi.service_calls":0,"queuetimems":700,"class":"MiniHBaseClusterRegionServer","multi.mutations":0} 2023-12-10 06:56:22,892 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702207592073","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":14000,"client":"172.31.14.131:41696","queuetimems":700,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:22,906 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":2,"starttimems":"1702207769373","responsesize":"20","method":"Multi","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., for 2 action(s) and 1st row key=t.TestQuotaAdmin0","processingtimems":12700,"client":"172.31.14.131:41696","multi.service_calls":0,"queuetimems":1200,"class":"MiniHBaseClusterRegionServer","multi.mutations":0} 2023-12-10 06:56:22,907 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702207812673","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":15400,"client":"172.31.14.131:41696","queuetimems":900,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:22,922 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":2,"starttimems":"1702207992773","responsesize":"20","method":"Multi","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., for 2 action(s) and 1st row key=t.TestQuotaAdmin0","processingtimems":12400,"client":"172.31.14.131:41696","multi.service_calls":0,"queuetimems":800,"class":"MiniHBaseClusterRegionServer","multi.mutations":0} 2023-12-10 06:56:22,924 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702208033573","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":15200,"client":"172.31.14.131:41696","queuetimems":200,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:22,927 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702208093373","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=exceedThrottleQuota","processingtimems":10500,"client":"172.31.14.131:41696","queuetimems":900,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:22,934 INFO [regionserver/jenkins-hbase4:0.Chore.1] regionserver.HRegionServer$PeriodicMemStoreFlusher(1921): MemstoreFlusherChore requesting flush of hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b. because dca2e6ecbeef49fb8929fd36dd74df8b/q has an old edit so flush to free WALs after random delay 277775 ms 2023-12-10 06:56:22,941 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702208248173","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":10100,"client":"172.31.14.131:41696","queuetimems":600,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:22,980 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702208863273","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":11000,"client":"172.31.14.131:41696","queuetimems":700,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:22,997 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":2,"starttimems":"1702209153973","responsesize":"20","method":"Multi","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., for 2 action(s) and 1st row key=t.TestQuotaAdmin0","processingtimems":12000,"client":"172.31.14.131:41696","multi.service_calls":0,"queuetimems":1100,"class":"MiniHBaseClusterRegionServer","multi.mutations":0} 2023-12-10 06:56:22,998 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702209199973","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":13400,"client":"172.31.14.131:41696","queuetimems":800,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:23,008 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":2,"starttimems":"1702209340073","responsesize":"20","method":"Multi","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., for 2 action(s) and 1st row key=t.TestQuotaAdmin0","processingtimems":11300,"client":"172.31.14.131:41696","multi.service_calls":0,"queuetimems":500,"class":"MiniHBaseClusterRegionServer","multi.mutations":0} 2023-12-10 06:56:23,009 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702209376573","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":14800,"client":"172.31.14.131:41696","queuetimems":500,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:23,037 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702209846173","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":12000,"client":"172.31.14.131:41696","queuetimems":700,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:23,042 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2489): Flush column family: q of dca2e6ecbeef49fb8929fd36dd74df8b because time of oldest edit=1702198560273 is > 3600000 from now =1702209919173 2023-12-10 06:56:23,042 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing dca2e6ecbeef49fb8929fd36dd74df8b 1/2 column families, dataSize=116 B heapSize=776 B; q={dataSize=116 B, heapSize=520 B, offHeapSize=0 B} 2023-12-10 06:56:23,045 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":2,"starttimems":"1702209983873","responsesize":"20","method":"Multi","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., for 2 action(s) and 1st row key=t.TestQuotaAdmin0","processingtimems":10800,"client":"172.31.14.131:41696","multi.service_calls":0,"queuetimems":1000,"class":"MiniHBaseClusterRegionServer","multi.mutations":0} 2023-12-10 06:56:23,047 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702210019673","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":12200,"client":"172.31.14.131:41696","queuetimems":800,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:23,064 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702210164873","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":11700,"client":"172.31.14.131:41696","queuetimems":500,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:23,066 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=116 B at sequenceid=14 (bloomFilter=true), to=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/.tmp/q/01f660fa9b55470f95c556f0b2cb2707 2023-12-10 06:56:23,076 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702210320773","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":13600,"client":"172.31.14.131:41696","queuetimems":700,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:23,080 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/.tmp/q/01f660fa9b55470f95c556f0b2cb2707 as hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/q/01f660fa9b55470f95c556f0b2cb2707 2023-12-10 06:56:23,088 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":2,"starttimems":"1702210482973","responsesize":"20","method":"Multi","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., for 2 action(s) and 1st row key=t.TestQuotaAdmin0","processingtimems":13900,"client":"172.31.14.131:41696","multi.service_calls":0,"queuetimems":500,"class":"MiniHBaseClusterRegionServer","multi.mutations":0} 2023-12-10 06:56:23,088 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/q/01f660fa9b55470f95c556f0b2cb2707, entries=1, sequenceid=14, filesize=4.8 K 2023-12-10 06:56:23,089 WARN [AsyncFSWAL-0-hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2-prefix:jenkins-hbase4.apache.org,37997,1702191353702] wal.MetricsWAL(65): AsyncFSWAL-0-hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2-prefix:jenkins-hbase4.apache.org,37997,1702191353702 took 3300 ms appending an edit to wal; len~=202 2023-12-10 06:56:23,090 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~116 B/116, heapSize ~504 B/504, currentSize=0 B/0 for dca2e6ecbeef49fb8929fd36dd74df8b in 623900ms, sequenceid=14, compaction requested=false 2023-12-10 06:56:23,091 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for dca2e6ecbeef49fb8929fd36dd74df8b: 2023-12-10 06:56:23,095 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702210525273","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":155700,"client":"172.31.14.131:41696","queuetimems":700,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:23,109 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702210843573","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":13900,"client":"172.31.14.131:41696","queuetimems":500,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:23,119 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702211003773","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":13200,"client":"172.31.14.131:41696","queuetimems":600,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:23,130 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702211173973","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":10300,"client":"172.31.14.131:41696","queuetimems":500,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:23,138 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702211326273","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":12700,"client":"172.31.14.131:41696","queuetimems":700,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:23,245 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702213065773","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":11400,"client":"172.31.14.131:41696","queuetimems":400,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:23,255 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702213220673","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":11400,"client":"172.31.14.131:41696","queuetimems":400,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:23,277 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702213586473","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":10600,"client":"172.31.14.131:41696","queuetimems":600,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:23,362 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702215165673","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":12100,"client":"172.31.14.131:41696","queuetimems":800,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:23,634 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702220212173","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":11600,"client":"172.31.14.131:41696","queuetimems":700,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:23,653 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702220572073","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":10300,"client":"172.31.14.131:41696","queuetimems":800,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:23,679 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702220964573","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":11100,"client":"172.31.14.131:41696","queuetimems":700,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:23,726 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702221866473","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":10500,"client":"172.31.14.131:41696","queuetimems":700,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:23,732 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702221992273","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":11600,"client":"172.31.14.131:41696","queuetimems":800,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:23,801 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702223316373","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":10200,"client":"172.31.14.131:41696","queuetimems":600,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:23,989 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702226653473","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":11700,"client":"172.31.14.131:41696","queuetimems":800,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:23,996 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702226789173","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":11300,"client":"172.31.14.131:41696","queuetimems":600,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:24,003 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702226905173","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":10400,"client":"172.31.14.131:41696","queuetimems":500,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:24,047 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702227620873","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":11600,"client":"172.31.14.131:41696","queuetimems":800,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:24,053 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702227740673","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":10700,"client":"172.31.14.131:41696","queuetimems":900,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:24,129 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702229242273","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":11300,"client":"172.31.14.131:41696","queuetimems":900,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:24,668 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702238862073","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":14000,"client":"172.31.14.131:41696","queuetimems":800,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:24,674 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1702238997473","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":10300,"client":"172.31.14.131:41696","queuetimems":700,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:56:25,174 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] ipc.RpcServer(528): (responseTooSlow): {"call":"GetClusterStatus(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$GetClusterStatusRequest)","starttimems":"1702248216973","responsesize":"339","method":"GetClusterStatus","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$GetClusterStatusRequest","processingtimems":23500,"client":"172.31.14.131:41560","queuetimems":600,"class":"HMaster"} 2023-12-10 06:56:26,898 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] ipc.RpcServer(528): (responseTooSlow): {"call":"GetClusterStatus(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$GetClusterStatusRequest)","starttimems":"1702280440073","responsesize":"339","method":"GetClusterStatus","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$GetClusterStatusRequest","processingtimems":19900,"client":"172.31.14.131:41560","queuetimems":500,"class":"HMaster"} 2023-12-10 06:56:28,280 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.CallRunner(105): Dropping timed out call: callId: 7576 service: ClientService methodName: Get size: 134 connection: 172.31.14.131:41696 deadline: 1702304898273 param: region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=exceedThrottleQuota connection: 172.31.14.131:41696 2023-12-10 06:56:28,723 WARN [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=40533] ipc.RpcServer(528): (responseTooSlow): {"call":"RegionServerReport(org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerReportRequest)","starttimems":"1702320996773","responsesize":"0","method":"RegionServerReport","param":"server host_name: \"jenkins-hbase4.apache.org\" port: 37997 start_code: 1702191353702 load { numberOfRequests: 0 }","processingtimems":36800,"client":"172.31.14.131:57329","queuetimems":400,"class":"HMaster"} 2023-12-10 06:56:34,092 WARN [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=40533] ipc.RpcServer(528): (responseTooSlow): {"call":"RegionServerReport(org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerReportRequest)","starttimems":"1702534602673","responsesize":"0","method":"RegionServerReport","param":"server host_name: \"jenkins-hbase4.apache.org\" port: 42073 start_code: 1702191353745 load { numberOfRequests: 0 }","processingtimems":15000,"client":"172.31.14.131:33643","queuetimems":700,"class":"HMaster"} 2023-12-10 06:56:54,793 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(142): Chore: MemstoreFlusherChore missed its start time 2023-12-10 06:56:54,794 INFO [regionserver/jenkins-hbase4:0.Chore.2] hbase.ScheduledChore(142): Chore: CompactionChecker missed its start time 2023-12-10 06:56:54,796 WARN [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=40533] ipc.RpcServer(528): (responseTooSlow): {"call":"RegionServerReport(org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerReportRequest)","starttimems":"1703157381373","responsesize":"0","method":"RegionServerReport","param":"server host_name: \"jenkins-hbase4.apache.org\" port: 37997 start_code: 1702191353702 load { numberOfRequests: 0 }","processingtimems":14000,"client":"172.31.14.131:57329","queuetimems":0,"class":"HMaster"} 2023-12-10 06:56:54,798 INFO [JvmPauseMonitor] util.JvmPauseMonitor$Monitor(172): Detected pause in JVM or host machine (eg GC): pause of approximately 4185ms GC pool 'PS MarkSweep' had collection(s): count=1 time=4272ms GC pool 'PS Scavenge' had collection(s): count=1 time=159ms 2023-12-10 06:56:54,798 INFO [JvmPauseMonitor] util.JvmPauseMonitor$Monitor(172): Detected pause in JVM or host machine (eg GC): pause of approximately 4185ms GC pool 'PS MarkSweep' had collection(s): count=1 time=4272ms GC pool 'PS Scavenge' had collection(s): count=1 time=159ms 2023-12-10 06:57:14,184 INFO [regionserver/jenkins-hbase4:0.Chore.2] hbase.ScheduledChore(142): Chore: CompactionChecker missed its start time 2023-12-10 06:57:14,184 INFO [regionserver/jenkins-hbase4:0.Chore.2] hbase.ScheduledChore(142): Chore: MemstoreFlusherChore missed its start time 2023-12-10 06:57:14,185 INFO [JvmPauseMonitor] util.JvmPauseMonitor$Monitor(172): Detected pause in JVM or host machine (eg GC): pause of approximately 5047ms GC pool 'PS MarkSweep' had collection(s): count=1 time=4819ms GC pool 'PS Scavenge' had collection(s): count=1 time=432ms 2023-12-10 06:57:14,185 INFO [JvmPauseMonitor] util.JvmPauseMonitor$Monitor(172): Detected pause in JVM or host machine (eg GC): pause of approximately 5047ms GC pool 'PS MarkSweep' had collection(s): count=1 time=4819ms GC pool 'PS Scavenge' had collection(s): count=1 time=432ms 2023-12-10 06:57:14,197 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Scan(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ScanRequest)","starttimems":"1703654114173","responsesize":"92","method":"Scan","param":"region { type: REGION_NAME value: \"hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b.\" } scan { column { family: \"q\" } time_range { from: 0 ","processingtimems":28500,"client":"172.31.14.131:41678","queuetimems":700,"class":"MiniHBaseClusterRegionServer","scandetails":"table: hbase:quota region: hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b."} 2023-12-10 06:57:26,170 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(142): Chore: CompactionChecker missed its start time 2023-12-10 06:57:26,170 INFO [regionserver/jenkins-hbase4:0.Chore.3] hbase.ScheduledChore(142): Chore: MemstoreFlusherChore missed its start time 2023-12-10 06:57:26,170 INFO [JvmPauseMonitor] util.JvmPauseMonitor$Monitor(172): Detected pause in JVM or host machine (eg GC): pause of approximately 9985ms GC pool 'PS MarkSweep' had collection(s): count=1 time=10192ms 2023-12-10 06:57:26,170 INFO [JvmPauseMonitor] util.JvmPauseMonitor$Monitor(172): Detected pause in JVM or host machine (eg GC): pause of approximately 9985ms GC pool 'PS MarkSweep' had collection(s): count=1 time=10192ms 2023-12-10 06:57:26,171 WARN [RS:1;jenkins-hbase4:42073] util.Sleeper(86): We slept 10276ms instead of 100ms, this is likely due to a long garbage collecting pause and it's usually bad, see http://hbase.apache.org/book.html#trouble.rs.runtime.zkexpired 2023-12-10 06:57:26,171 WARN [M:0;jenkins-hbase4:40533] util.Sleeper(86): We slept 10284ms instead of 100ms, this is likely due to a long garbage collecting pause and it's usually bad, see http://hbase.apache.org/book.html#trouble.rs.runtime.zkexpired 2023-12-10 06:57:26,171 WARN [RS:0;jenkins-hbase4:37997] util.Sleeper(86): We slept 10276ms instead of 100ms, this is likely due to a long garbage collecting pause and it's usually bad, see http://hbase.apache.org/book.html#trouble.rs.runtime.zkexpired 2023-12-10 06:57:26,181 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] ipc.RpcServer(528): (responseTooSlow): {"call":"ReportRegionSpaceUse(org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionSpaceUseReportRequest)","starttimems":"1703727417073","responsesize":"0","method":"ReportRegionSpaceUse","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionSpaceUseReportRequest","processingtimems":61900,"client":"172.31.14.131:57329","queuetimems":1200,"class":"HMaster"} 2023-12-10 06:57:34,109 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(142): Chore: CompactionChecker missed its start time 2023-12-10 06:57:34,109 INFO [regionserver/jenkins-hbase4:0.Chore.2] hbase.ScheduledChore(142): Chore: MemstoreFlusherChore missed its start time 2023-12-10 06:57:34,110 INFO [JvmPauseMonitor] util.JvmPauseMonitor$Monitor(172): Detected pause in JVM or host machine (eg GC): pause of approximately 5938ms GC pool 'PS MarkSweep' had collection(s): count=1 time=6172ms 2023-12-10 06:57:34,110 INFO [JvmPauseMonitor] util.JvmPauseMonitor$Monitor(172): Detected pause in JVM or host machine (eg GC): pause of approximately 5939ms GC pool 'PS MarkSweep' had collection(s): count=1 time=6172ms 2023-12-10 06:57:34,119 WARN [regionserver/jenkins-hbase4:0.Chore.1] quotas.QuotaCache$QuotaRefresherChore(300): Unable to read if exceed throttle quota enabled from quota table java.net.SocketTimeoutException: callTimeout=1200000, callDuration=1494749850: Call to address=jenkins-hbase4.apache.org:37997 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=7576,methodName=Get], waitTime=1494736400ms, rpcTimeout=60000ms row 'exceedThrottleQuota' on table 'hbase:quota' at region=hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., hostname=jenkins-hbase4.apache.org,37997,1702191353702, seqNum=2 at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:156) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:370) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:343) at org.apache.hadoop.hbase.quotas.QuotaTableUtil.doGet(QuotaTableUtil.java:903) at org.apache.hadoop.hbase.quotas.QuotaUtil.isExceedThrottleQuotaEnabled(QuotaUtil.java:233) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetchExceedThrottleQuota(QuotaCache.java:298) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.chore(QuotaCache.java:229) at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:107) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to address=jenkins-hbase4.apache.org:37997 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=7576,methodName=Get], waitTime=1494736400ms, rpcTimeout=60000ms at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:219) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:384) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:87) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:415) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:411) at org.apache.hadoop.hbase.ipc.Call.setTimeout(Call.java:108) at org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:134) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ... 1 more Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=7576,methodName=Get], waitTime=1494736400ms, rpcTimeout=60000ms at org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:135) ... 6 more 2023-12-10 06:57:36,169 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(142): Chore: CompactionChecker missed its start time 2023-12-10 06:57:36,185 INFO [regionserver/jenkins-hbase4:0.Chore.2] hbase.ScheduledChore(142): Chore: MemstoreFlusherChore missed its start time 2023-12-10 06:58:30,354 INFO [regionserver/jenkins-hbase4:0.Chore.3] hbase.ScheduledChore(142): Chore: CompactionThroughputTuner missed its start time 2023-12-10 06:58:30,475 INFO [regionserver/jenkins-hbase4:0.Chore.3] hbase.ScheduledChore(142): Chore: jenkins-hbase4.apache.org,42073,1702191353745-HeapMemoryTunerChore missed its start time 2023-12-10 06:58:59,066 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(142): Chore: CompactionChecker missed its start time 2023-12-10 06:58:59,066 INFO [regionserver/jenkins-hbase4:0.Chore.2] hbase.ScheduledChore(142): Chore: MemstoreFlusherChore missed its start time 2023-12-10 06:59:05,989 INFO [regionserver/jenkins-hbase4:0.Chore.5] hbase.ScheduledChore(142): Chore: SpaceQuotaRefresherChore missed its start time 2023-12-10 06:59:12,300 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(142): Chore: RegionSizeReportingChore missed its start time 2023-12-10 06:59:18,732 INFO [regionserver/jenkins-hbase4:0.Chore.6] hbase.ScheduledChore(142): Chore: MemstoreFlusherChore missed its start time 2023-12-10 06:59:18,732 INFO [regionserver/jenkins-hbase4:0.Chore.4] hbase.ScheduledChore(142): Chore: CompactionChecker missed its start time 2023-12-10 06:59:19,047 INFO [regionserver/jenkins-hbase4:0.Chore.4] hbase.ScheduledChore(142): Chore: CompactedHFilesCleaner missed its start time 2023-12-10 06:59:20,896 INFO [regionserver/jenkins-hbase4:0.Chore.3] hbase.ScheduledChore(142): Chore: MemstoreFlusherChore missed its start time 2023-12-10 06:59:20,899 INFO [regionserver/jenkins-hbase4:0.Chore.8] hbase.ScheduledChore(142): Chore: CompactionChecker missed its start time 2023-12-10 06:59:20,911 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59536, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=MasterService 2023-12-10 06:59:20,918 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35514, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=ClientService 2023-12-10 06:59:20,927 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Scan(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ScanRequest)","starttimems":"1703829149573","responsesize":"24","method":"Scan","param":"region { type: REGION_NAME value: \"hbase:meta,,1\" } scan { column { family: \"info\" } start_row: \"hbase:quota,n.TestNs,99999999999999\" stop_row: \"hbase ","processingtimems":252800,"client":"172.31.14.131:35514","queuetimems":0,"class":"MiniHBaseClusterRegionServer","scandetails":"table: hbase:meta region: hbase:meta,,1.1588230740"} 2023-12-10 06:59:20,929 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Scan(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ScanRequest)","starttimems":"1703829463673","responsesize":"735","method":"Scan","param":"scanner_id: 8252208067836379172 number_of_rows: 1 close_scanner: false next_call_seq: 0 client_handles_partials: true client_handles_heartbeats: true ","processingtimems":36200,"client":"172.31.14.131:35514","queuetimems":800,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:59:20,932 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1703829542773","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=n.TestNs","processingtimems":17100,"client":"172.31.14.131:35514","queuetimems":600,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:59:20,932 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1703829545773","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=n.TestNs","processingtimems":14300,"client":"172.31.14.131:35514","queuetimems":1800,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:59:20,960 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1703829808273","responsesize":"4","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=n.TestNs","processingtimems":39300,"client":"172.31.14.131:35514","queuetimems":3500,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 06:59:20,968 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.CallRunner(105): Dropping timed out call: callId: 7664 service: ClientService methodName: Get size: 117 connection: 172.31.14.131:35514 deadline: 1703829917173 param: region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=r.all connection: 172.31.14.131:35514 2023-12-10 06:59:21,019 WARN [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=40533] ipc.RpcServer(528): (responseTooSlow): {"call":"GetClusterStatus(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$GetClusterStatusRequest)","starttimems":"1703830557073","responsesize":"339","method":"GetClusterStatus","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$GetClusterStatusRequest","processingtimems":10600,"client":"172.31.14.131:59536","queuetimems":2300,"class":"HMaster"} 2023-12-10 06:59:21,107 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.CallRunner(105): Dropping timed out call: callId: 8436 service: ClientService methodName: Get size: 156 connection: 172.31.14.131:35514 deadline: 1703831817573 param: region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins connection: 172.31.14.131:35514 2023-12-10 06:59:21,108 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.CallRunner(105): Dropping timed out call: callId: 8437 service: ClientService methodName: Get size: 117 connection: 172.31.14.131:35514 deadline: 1703831817573 param: region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=r.all connection: 172.31.14.131:35514 2023-12-10 06:59:21,108 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.CallRunner(105): Dropping timed out call: callId: 8438 service: ClientService methodName: Get size: 134 connection: 172.31.14.131:35514 deadline: 1703831817573 param: region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=exceedThrottleQuota connection: 172.31.14.131:35514 2023-12-10 06:59:21,108 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.CallRunner(105): Dropping timed out call: callId: 8439 service: ClientService methodName: Get size: 117 connection: 172.31.14.131:35514 deadline: 1703831817573 param: region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=r.all connection: 172.31.14.131:35514 2023-12-10 06:59:21,108 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.CallRunner(105): Dropping timed out call: callId: 8440 service: ClientService methodName: Get size: 134 connection: 172.31.14.131:35514 deadline: 1703831817573 param: region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=exceedThrottleQuota connection: 172.31.14.131:35514 2023-12-10 06:59:21,108 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.CallRunner(105): Dropping timed out call: callId: 8441 service: ClientService methodName: Get size: 156 connection: 172.31.14.131:35514 deadline: 1703831817573 param: region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins connection: 172.31.14.131:35514 2023-12-10 06:59:21,108 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.CallRunner(105): Dropping timed out call: callId: 8444 service: ClientService methodName: Get size: 120 connection: 172.31.14.131:35514 deadline: 1703831850573 param: region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=n.TestNs connection: 172.31.14.131:35514 2023-12-10 06:59:22,555 WARN [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=40533] ipc.CallRunner(105): Dropping timed out call: callId: 1774 service: RegionServerStatusService methodName: RegionServerReport size: 1.2 K connection: 172.31.14.131:57329 deadline: 1703875231473 param: server host_name: "jenkins-hbase4.apache.org" port: 37997 start_code: 1702191353702 load { numberOfRequests: 0 } connection: 172.31.14.131:57329 2023-12-10 06:59:38,603 INFO [regionserver/jenkins-hbase4:0.Chore.2] hbase.ScheduledChore(142): Chore: MemstoreFlusherChore missed its start time 2023-12-10 06:59:38,604 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(142): Chore: CompactionChecker missed its start time 2023-12-10 07:00:10,001 INFO [regionserver/jenkins-hbase4:0.Chore.3] hbase.ScheduledChore(142): Chore: MemstoreFlusherChore missed its start time 2023-12-10 07:00:10,001 INFO [JvmPauseMonitor] util.JvmPauseMonitor$Monitor(172): Detected pause in JVM or host machine (eg GC): pause of approximately 6336ms GC pool 'PS MarkSweep' had collection(s): count=1 time=5474ms GC pool 'PS Scavenge' had collection(s): count=1 time=1004ms 2023-12-10 07:00:10,001 INFO [JvmPauseMonitor] util.JvmPauseMonitor$Monitor(172): Detected pause in JVM or host machine (eg GC): pause of approximately 6336ms GC pool 'PS MarkSweep' had collection(s): count=1 time=5474ms GC pool 'PS Scavenge' had collection(s): count=1 time=1004ms 2023-12-10 07:00:10,001 INFO [regionserver/jenkins-hbase4:0.Chore.2] hbase.ScheduledChore(142): Chore: CompactionChecker missed its start time 2023-12-10 07:00:11,262 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Scan(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ScanRequest)","starttimems":"1705121150173","responsesize":"92","method":"Scan","param":"region { type: REGION_NAME value: \"hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b.\" } scan { column { family: \"q\" } time_range { from: 0 ","processingtimems":16100,"client":"172.31.14.131:41678","queuetimems":1000,"class":"MiniHBaseClusterRegionServer","scandetails":"table: hbase:quota region: hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b."} 2023-12-10 07:00:18,178 INFO [JvmPauseMonitor] util.JvmPauseMonitor$Monitor(172): Detected pause in JVM or host machine (eg GC): pause of approximately 4157ms GC pool 'PS MarkSweep' had collection(s): count=1 time=3912ms GC pool 'PS Scavenge' had collection(s): count=1 time=434ms 2023-12-10 07:00:18,178 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(142): Chore: MemstoreFlusherChore missed its start time 2023-12-10 07:00:18,178 INFO [JvmPauseMonitor] util.JvmPauseMonitor$Monitor(172): Detected pause in JVM or host machine (eg GC): pause of approximately 4157ms GC pool 'PS MarkSweep' had collection(s): count=1 time=3912ms GC pool 'PS Scavenge' had collection(s): count=1 time=434ms 2023-12-10 07:00:18,178 INFO [regionserver/jenkins-hbase4:0.Chore.2] hbase.ScheduledChore(142): Chore: CompactionChecker missed its start time 2023-12-10 07:00:29,747 INFO [regionserver/jenkins-hbase4:0.Chore.2] hbase.ScheduledChore(142): Chore: MemstoreFlusherChore missed its start time 2023-12-10 07:00:29,747 INFO [JvmPauseMonitor] util.JvmPauseMonitor$Monitor(172): Detected pause in JVM or host machine (eg GC): pause of approximately 9568ms GC pool 'PS MarkSweep' had collection(s): count=1 time=9746ms 2023-12-10 07:00:29,747 INFO [JvmPauseMonitor] util.JvmPauseMonitor$Monitor(172): Detected pause in JVM or host machine (eg GC): pause of approximately 9567ms GC pool 'PS MarkSweep' had collection(s): count=1 time=9746ms 2023-12-10 07:00:29,749 INFO [regionserver/jenkins-hbase4:0.Chore.3] hbase.ScheduledChore(142): Chore: CompactionChecker missed its start time 2023-12-10 07:00:29,753 WARN [regionserver/jenkins-hbase4:0.Chore.3] quotas.QuotaCache$QuotaRefresherChore(344): Unable to read regionServer from quota table java.net.SocketTimeoutException: callTimeout=1200000, callDuration=1461565251: Call to address=jenkins-hbase4.apache.org:37997 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=7664,methodName=Get], waitTime=1461536900ms, rpcTimeout=60000ms row 'r.all' on table 'hbase:quota' at region=hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., hostname=jenkins-hbase4.apache.org,37997,1702191353702, seqNum=2 at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:156) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:370) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:343) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:385) at org.apache.hadoop.hbase.quotas.QuotaTableUtil.doGet(QuotaTableUtil.java:910) at org.apache.hadoop.hbase.quotas.QuotaUtil.fetchGlobalQuotas(QuotaUtil.java:374) at org.apache.hadoop.hbase.quotas.QuotaUtil.fetchRegionServerQuotas(QuotaUtil.java:357) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore$4.fetchEntries(QuotaCache.java:290) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetch(QuotaCache.java:333) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetchRegionServerQuotaState(QuotaCache.java:281) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.chore(QuotaCache.java:228) at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:107) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to address=jenkins-hbase4.apache.org:37997 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=7664,methodName=Get], waitTime=1461536900ms, rpcTimeout=60000ms at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:219) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:384) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:87) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:415) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:411) at org.apache.hadoop.hbase.ipc.Call.setTimeout(Call.java:108) at org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:134) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ... 1 more Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=7664,methodName=Get], waitTime=1461536900ms, rpcTimeout=60000ms at org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:135) ... 6 more 2023-12-10 07:00:29,755 WARN [regionserver/jenkins-hbase4:0.Chore.8] quotas.QuotaCache$QuotaRefresherChore(344): Unable to read user from quota table java.net.SocketTimeoutException: callTimeout=1200000, callDuration=1459669952: Call to address=jenkins-hbase4.apache.org:37997 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=8441,methodName=Get], waitTime=1459658900ms, rpcTimeout=60000ms row 'u.jenkins' on table 'hbase:quota' at region=hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., hostname=jenkins-hbase4.apache.org,37997,1702191353702, seqNum=2 at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:156) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:370) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:343) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:385) at org.apache.hadoop.hbase.quotas.QuotaTableUtil.doGet(QuotaTableUtil.java:910) at org.apache.hadoop.hbase.quotas.QuotaUtil.fetchUserQuotas(QuotaUtil.java:278) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore$3.fetchEntries(QuotaCache.java:274) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetch(QuotaCache.java:333) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetchUserQuotaState(QuotaCache.java:266) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.chore(QuotaCache.java:227) at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:107) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to address=jenkins-hbase4.apache.org:37997 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=8441,methodName=Get], waitTime=1459658900ms, rpcTimeout=60000ms at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:219) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:384) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:87) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:415) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:411) at org.apache.hadoop.hbase.ipc.Call.setTimeout(Call.java:108) at org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:134) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ... 1 more Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=8441,methodName=Get], waitTime=1459658900ms, rpcTimeout=60000ms at org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:135) ... 6 more 2023-12-10 07:00:29,756 WARN [regionserver/jenkins-hbase4:0.Chore.6] quotas.QuotaCache$QuotaRefresherChore(300): Unable to read if exceed throttle quota enabled from quota table java.net.SocketTimeoutException: callTimeout=1200000, callDuration=1459670050: Call to address=jenkins-hbase4.apache.org:37997 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=8438,methodName=Get], waitTime=1459656700ms, rpcTimeout=60000ms row 'exceedThrottleQuota' on table 'hbase:quota' at region=hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., hostname=jenkins-hbase4.apache.org,37997,1702191353702, seqNum=2 at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:156) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:370) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:343) at org.apache.hadoop.hbase.quotas.QuotaTableUtil.doGet(QuotaTableUtil.java:903) at org.apache.hadoop.hbase.quotas.QuotaUtil.isExceedThrottleQuotaEnabled(QuotaUtil.java:233) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetchExceedThrottleQuota(QuotaCache.java:298) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.chore(QuotaCache.java:229) at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:107) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to address=jenkins-hbase4.apache.org:37997 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=8438,methodName=Get], waitTime=1459656700ms, rpcTimeout=60000ms at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:219) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:384) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:87) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:415) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:411) at org.apache.hadoop.hbase.ipc.Call.setTimeout(Call.java:108) at org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:134) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ... 1 more Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=8438,methodName=Get], waitTime=1459656700ms, rpcTimeout=60000ms at org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:135) ... 6 more 2023-12-10 07:00:29,756 WARN [regionserver/jenkins-hbase4:0.Chore.5] quotas.QuotaCache$QuotaRefresherChore(344): Unable to read regionServer from quota table java.net.SocketTimeoutException: callTimeout=1200000, callDuration=1459672052: Call to address=jenkins-hbase4.apache.org:37997 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=8437,methodName=Get], waitTime=1459655000ms, rpcTimeout=60000ms row 'r.all' on table 'hbase:quota' at region=hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., hostname=jenkins-hbase4.apache.org,37997,1702191353702, seqNum=2 at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:156) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:370) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:343) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:385) at org.apache.hadoop.hbase.quotas.QuotaTableUtil.doGet(QuotaTableUtil.java:910) at org.apache.hadoop.hbase.quotas.QuotaUtil.fetchGlobalQuotas(QuotaUtil.java:374) at org.apache.hadoop.hbase.quotas.QuotaUtil.fetchRegionServerQuotas(QuotaUtil.java:357) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore$4.fetchEntries(QuotaCache.java:290) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetch(QuotaCache.java:333) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetchRegionServerQuotaState(QuotaCache.java:281) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.chore(QuotaCache.java:228) at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:107) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to address=jenkins-hbase4.apache.org:37997 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=8437,methodName=Get], waitTime=1459655000ms, rpcTimeout=60000ms at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:219) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:384) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:87) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:415) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:411) at org.apache.hadoop.hbase.ipc.Call.setTimeout(Call.java:108) at org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:134) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ... 1 more Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=8437,methodName=Get], waitTime=1459655000ms, rpcTimeout=60000ms at org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:135) ... 6 more 2023-12-10 07:00:29,755 WARN [regionserver/jenkins-hbase4:0.Chore.1] quotas.QuotaCache$QuotaRefresherChore(344): Unable to read namespace from quota table java.net.SocketTimeoutException: callTimeout=1200000, callDuration=1459645550: Call to address=jenkins-hbase4.apache.org:37997 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=8444,methodName=Get], waitTime=1459630900ms, rpcTimeout=60000ms row 'n.TestNs' on table 'hbase:quota' at region=hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., hostname=jenkins-hbase4.apache.org,37997,1702191353702, seqNum=2 at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:156) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:370) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:343) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:385) at org.apache.hadoop.hbase.quotas.QuotaTableUtil.doGet(QuotaTableUtil.java:910) at org.apache.hadoop.hbase.quotas.QuotaUtil.fetchGlobalQuotas(QuotaUtil.java:374) at org.apache.hadoop.hbase.quotas.QuotaUtil.fetchNamespaceQuotas(QuotaUtil.java:341) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore$1.fetchEntries(QuotaCache.java:242) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetch(QuotaCache.java:333) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetchNamespaceQuotaState(QuotaCache.java:234) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.chore(QuotaCache.java:225) at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:107) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to address=jenkins-hbase4.apache.org:37997 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=8444,methodName=Get], waitTime=1459630900ms, rpcTimeout=60000ms at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:219) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:384) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:87) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:415) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:411) at org.apache.hadoop.hbase.ipc.Call.setTimeout(Call.java:108) at org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:134) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ... 1 more Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=8444,methodName=Get], waitTime=1459630900ms, rpcTimeout=60000ms at org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:135) ... 6 more 2023-12-10 07:00:29,755 WARN [regionserver/jenkins-hbase4:0.Chore.4] quotas.QuotaCache$QuotaRefresherChore(344): Unable to read user from quota table java.net.SocketTimeoutException: callTimeout=1200000, callDuration=1459674152: Call to address=jenkins-hbase4.apache.org:37997 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=8436,methodName=Get], waitTime=1459652200ms, rpcTimeout=60000ms row 'u.jenkins' on table 'hbase:quota' at region=hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., hostname=jenkins-hbase4.apache.org,37997,1702191353702, seqNum=2 at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:156) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:370) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:343) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:385) at org.apache.hadoop.hbase.quotas.QuotaTableUtil.doGet(QuotaTableUtil.java:910) at org.apache.hadoop.hbase.quotas.QuotaUtil.fetchUserQuotas(QuotaUtil.java:278) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore$3.fetchEntries(QuotaCache.java:274) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetch(QuotaCache.java:333) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetchUserQuotaState(QuotaCache.java:266) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.chore(QuotaCache.java:227) at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:107) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to address=jenkins-hbase4.apache.org:37997 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=8436,methodName=Get], waitTime=1459652200ms, rpcTimeout=60000ms at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:219) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:384) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:87) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:415) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:411) at org.apache.hadoop.hbase.ipc.Call.setTimeout(Call.java:108) at org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:134) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ... 1 more Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=8436,methodName=Get], waitTime=1459652200ms, rpcTimeout=60000ms at org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:135) ... 6 more 2023-12-10 07:00:29,755 WARN [regionserver/jenkins-hbase4:0.Chore.7] quotas.QuotaCache$QuotaRefresherChore(300): Unable to read if exceed throttle quota enabled from quota table java.net.SocketTimeoutException: callTimeout=1200000, callDuration=1459669951: Call to address=jenkins-hbase4.apache.org:37997 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=8440,methodName=Get], waitTime=1459658400ms, rpcTimeout=60000ms row 'exceedThrottleQuota' on table 'hbase:quota' at region=hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., hostname=jenkins-hbase4.apache.org,37997,1702191353702, seqNum=2 at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:156) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:370) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:343) at org.apache.hadoop.hbase.quotas.QuotaTableUtil.doGet(QuotaTableUtil.java:903) at org.apache.hadoop.hbase.quotas.QuotaUtil.isExceedThrottleQuotaEnabled(QuotaUtil.java:233) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetchExceedThrottleQuota(QuotaCache.java:298) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.chore(QuotaCache.java:229) at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:107) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to address=jenkins-hbase4.apache.org:37997 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=8440,methodName=Get], waitTime=1459658400ms, rpcTimeout=60000ms at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:219) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:384) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:87) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:415) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:411) at org.apache.hadoop.hbase.ipc.Call.setTimeout(Call.java:108) at org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:134) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ... 1 more Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=8440,methodName=Get], waitTime=1459658400ms, rpcTimeout=60000ms at org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:135) ... 6 more 2023-12-10 07:00:29,755 WARN [regionserver/jenkins-hbase4:0.Chore.2] quotas.QuotaCache$QuotaRefresherChore(344): Unable to read regionServer from quota table java.net.SocketTimeoutException: callTimeout=1200000, callDuration=1459670050: Call to address=jenkins-hbase4.apache.org:37997 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=8439,methodName=Get], waitTime=1459657800ms, rpcTimeout=60000ms row 'r.all' on table 'hbase:quota' at region=hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., hostname=jenkins-hbase4.apache.org,37997,1702191353702, seqNum=2 at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:156) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:370) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:343) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:385) at org.apache.hadoop.hbase.quotas.QuotaTableUtil.doGet(QuotaTableUtil.java:910) at org.apache.hadoop.hbase.quotas.QuotaUtil.fetchGlobalQuotas(QuotaUtil.java:374) at org.apache.hadoop.hbase.quotas.QuotaUtil.fetchRegionServerQuotas(QuotaUtil.java:357) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore$4.fetchEntries(QuotaCache.java:290) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetch(QuotaCache.java:333) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetchRegionServerQuotaState(QuotaCache.java:281) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.chore(QuotaCache.java:228) at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:107) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call to address=jenkins-hbase4.apache.org:37997 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=8439,methodName=Get], waitTime=1459657800ms, rpcTimeout=60000ms at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:219) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:384) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:87) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:415) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:411) at org.apache.hadoop.hbase.ipc.Call.setTimeout(Call.java:108) at org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:134) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.run(HashedWheelTimer.java:715) at org.apache.hbase.thirdparty.io.netty.util.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:34) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:703) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$HashedWheelBucket.expireTimeouts(HashedWheelTimer.java:790) at org.apache.hbase.thirdparty.io.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:503) ... 1 more Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call[id=8439,methodName=Get], waitTime=1459657800ms, rpcTimeout=60000ms at org.apache.hadoop.hbase.ipc.RpcConnection$1.run(RpcConnection.java:135) ... 6 more 2023-12-10 07:00:32,197 INFO [regionserver/jenkins-hbase4:0.Chore.8] hbase.ScheduledChore(142): Chore: MemstoreFlusherChore missed its start time 2023-12-10 07:00:32,197 INFO [regionserver/jenkins-hbase4:0.Chore.4] hbase.ScheduledChore(142): Chore: CompactionChecker missed its start time 2023-12-10 07:00:53,718 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache(965): totalSize=782.40 MB, usedSize=602.02 KB, freeSize=781.81 MB, max=782.40 MB, blockCount=8, accesses=5949, hits=5938, hitRatio=99.82%, , cachingAccesses=5949, cachingHits=5938, cachingHitsRatio=99.82%, evictions=28, evicted=0, evictedPerRun=0.0 2023-12-10 07:00:53,750 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache(965): totalSize=782.40 MB, usedSize=587.81 KB, freeSize=781.83 MB, max=782.40 MB, blockCount=0, accesses=0, hits=0, hitRatio=0, cachingAccesses=0, cachingHits=0, cachingHitsRatio=0,evictions=28, evicted=0, evictedPerRun=0.0 2023-12-10 07:00:53,808 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics] regionserver.ChunkCreator$MemStoreChunkPool$StatisticsThread(426): data stats (chunk size=2097152): current pool size=5, created chunk count=6, reused chunk count=1, reuseRatio=14.29% 2023-12-10 07:00:53,808 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MemStoreChunkPool Statistics] regionserver.ChunkCreator$MemStoreChunkPool$StatisticsThread(426): index stats (chunk size=209715): current pool size=0, created chunk count=0, reused chunk count=0, reuseRatio=0 2023-12-10 07:00:54,973 INFO [jenkins-hbase4:37997Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-12-10 07:00:54,973 INFO [jenkins-hbase4:42073Replication Statistics #0] regionserver.Replication$ReplicationStatisticsTask(246): Global stats: WAL Edits Buffer Used=0B, Limit=268435456B 2023-12-10 07:00:55,640 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(200): Skipping table TestQuotaAdmin2 because normalization is disabled in its table properties and normalization is also disabled at table level by default 2023-12-10 07:00:55,640 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(200): Skipping table TestQuotaAdmin1 because normalization is disabled in its table properties and normalization is also disabled at table level by default 2023-12-10 07:00:55,640 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(200): Skipping table TestNs:TestTable because normalization is disabled in its table properties and normalization is also disabled at table level by default 2023-12-10 07:00:55,640 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(200): Skipping table TestQuotaAdmin0 because normalization is disabled in its table properties and normalization is also disabled at table level by default 2023-12-10 07:00:56,317 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-12-10 07:00:56,319 DEBUG [master/jenkins-hbase4:0.Chore.1] balancer.BaseLoadBalancer(1718): Start Generate Balance plan for cluster. 2023-12-10 07:00:56,320 DEBUG [master/jenkins-hbase4:0.Chore.1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-12-10 07:00:56,320 DEBUG [master/jenkins-hbase4:0.Chore.1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-12-10 07:00:56,320 DEBUG [master/jenkins-hbase4:0.Chore.1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-12-10 07:00:56,320 DEBUG [master/jenkins-hbase4:0.Chore.1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=7, number of hosts=1, number of racks=1 2023-12-10 07:00:56,323 INFO [master/jenkins-hbase4:0.Chore.1] balancer.StochasticLoadBalancer(389): Cluster wide - Calculating plan. may take up to 30000ms to complete. 2023-12-10 07:00:56,323 INFO [master/jenkins-hbase4:0.Chore.1] balancer.StochasticLoadBalancer(505): Start StochasticLoadBalancer.balancer, initial weighted average imbalance=0.6478405315614617, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.7499999999999999, need balance); PrimaryRegionCountSkewCostFunction : (not needed); MoveCostFunction : (multiplier=7.0, imbalance=0.0); ServerLocalityCostFunction : (multiplier=25.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (not needed); RegionReplicaRackCostFunction : (not needed); ReadRequestCostFunction : (multiplier=5.0, imbalance=1.0, need balance); WriteRequestCostFunction : (multiplier=5.0, imbalance=1.0, need balance); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=1.0, need balance); computedMaxSteps=12800 2023-12-10 07:00:56,493 INFO [master/jenkins-hbase4:0.Chore.1] balancer.StochasticLoadBalancer(553): Finished computing new moving plan. Computation took 9900 ms to try 12800 different iterations. Found a solution that moves 3 regions; Going from a computed imbalance of 0.6478405315614617 to a new imbalance of 0.01934674274158602. funtionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0); PrimaryRegionCountSkewCostFunction : (not needed); MoveCostFunction : (multiplier=7.0, imbalance=0.375, need balance); ServerLocalityCostFunction : (multiplier=25.0, imbalance=0.0); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0); TableSkewCostFunction : (multiplier=35.0, imbalance=0.0); RegionReplicaHostCostFunction : (not needed); RegionReplicaRackCostFunction : (not needed); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.8043478260869567, need balance); WriteRequestCostFunction : (multiplier=5.0, imbalance=1.0, need balance); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0); StoreFileCostFunction : (multiplier=5.0, imbalance=0.0); 2023-12-10 07:00:56,493 INFO [master/jenkins-hbase4:0.Chore.1] master.HMaster(1852): Balancer plans size is 3, the balance interval is 100000 ms, and the max number regions in transition is 8 2023-12-10 07:00:56,494 INFO [master/jenkins-hbase4:0.Chore.1] master.HMaster(1857): balance hri=738cc1ffb58c4f3e8fbde456163cdb37, source=jenkins-hbase4.apache.org,37997,1702191353702, destination=jenkins-hbase4.apache.org,42073,1702191353745 2023-12-10 07:00:56,495 DEBUG [master/jenkins-hbase4:0.Chore.1] procedure2.ProcedureExecutor(1075): Stored pid=27, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=TestQuotaAdmin0, region=738cc1ffb58c4f3e8fbde456163cdb37, REOPEN/MOVE 2023-12-10 07:00:56,497 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=27, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=TestQuotaAdmin0, region=738cc1ffb58c4f3e8fbde456163cdb37, REOPEN/MOVE 2023-12-10 07:00:56,498 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=27 updating hbase:meta row=738cc1ffb58c4f3e8fbde456163cdb37, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 07:00:56,498 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1705291892473"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1705291892473"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1705291892473"}]},"ts":"1705291892473"} 2023-12-10 07:00:56,501 INFO [PEWorker-2] procedure2.ProcedureExecutor(1796): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE; CloseRegionProcedure 738cc1ffb58c4f3e8fbde456163cdb37, server=jenkins-hbase4.apache.org,37997,1702191353702}] 2023-12-10 07:00:56,659 DEBUG [RSProcedureDispatcher-pool-3] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 07:00:56,659 DEBUG [RSProcedureDispatcher-pool-3] ipc.RpcConnection(122): Using SIMPLE authentication for service=AdminService, sasl=false 2023-12-10 07:00:56,660 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50758, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-12-10 07:00:56,661 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 738cc1ffb58c4f3e8fbde456163cdb37 2023-12-10 07:00:56,662 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 738cc1ffb58c4f3e8fbde456163cdb37, disabling compactions & flushes 2023-12-10 07:00:56,662 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37. 2023-12-10 07:00:56,662 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37. 2023-12-10 07:00:56,663 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37. after waiting 0 ms 2023-12-10 07:00:56,663 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37. 2023-12-10 07:00:56,670 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin0/738cc1ffb58c4f3e8fbde456163cdb37/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-12-10 07:00:56,672 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37. 2023-12-10 07:00:56,672 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 738cc1ffb58c4f3e8fbde456163cdb37: 2023-12-10 07:00:56,672 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3515): Adding 738cc1ffb58c4f3e8fbde456163cdb37 move to jenkins-hbase4.apache.org,42073,1702191353745 record at close sequenceid=2 2023-12-10 07:00:56,674 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 738cc1ffb58c4f3e8fbde456163cdb37 2023-12-10 07:00:56,675 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=27 updating hbase:meta row=738cc1ffb58c4f3e8fbde456163cdb37, regionState=CLOSED 2023-12-10 07:00:56,675 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1705291894673"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1705291894673"}]},"ts":"1705291894673"} 2023-12-10 07:00:56,678 INFO [PEWorker-4] procedure2.ProcedureExecutor(1953): Finished subprocedure pid=28, resume processing ppid=27 2023-12-10 07:00:56,678 INFO [PEWorker-4] procedure2.ProcedureExecutor(1457): Finished pid=28, ppid=27, state=SUCCESS; CloseRegionProcedure 738cc1ffb58c4f3e8fbde456163cdb37, server=jenkins-hbase4.apache.org,37997,1702191353702 in 2.2000 sec 2023-12-10 07:00:56,679 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestQuotaAdmin0, region=738cc1ffb58c4f3e8fbde456163cdb37, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,42073,1702191353745; forceNewPlan=false, retain=false 2023-12-10 07:00:56,829 INFO [jenkins-hbase4:40533] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-12-10 07:00:56,829 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=27 updating hbase:meta row=738cc1ffb58c4f3e8fbde456163cdb37, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42073,1702191353745 2023-12-10 07:00:56,830 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1705291896073"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1705291896073"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1705291896073"}]},"ts":"1705291896073"} 2023-12-10 07:00:56,831 INFO [PEWorker-1] procedure2.ProcedureExecutor(1796): Initialized subprocedures=[{pid=29, ppid=27, state=RUNNABLE; OpenRegionProcedure 738cc1ffb58c4f3e8fbde456163cdb37, server=jenkins-hbase4.apache.org,42073,1702191353745}] 2023-12-10 07:00:56,984 DEBUG [RSProcedureDispatcher-pool-4] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,42073,1702191353745 2023-12-10 07:00:56,984 DEBUG [RSProcedureDispatcher-pool-4] ipc.RpcConnection(122): Using SIMPLE authentication for service=AdminService, sasl=false 2023-12-10 07:00:56,986 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52238, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-12-10 07:00:56,992 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37. 2023-12-10 07:00:56,992 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 738cc1ffb58c4f3e8fbde456163cdb37, NAME => 'TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37.', STARTKEY => '', ENDKEY => ''} 2023-12-10 07:00:56,992 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestQuotaAdmin0 738cc1ffb58c4f3e8fbde456163cdb37 2023-12-10 07:00:56,993 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-12-10 07:00:56,993 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 738cc1ffb58c4f3e8fbde456163cdb37 2023-12-10 07:00:56,993 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 738cc1ffb58c4f3e8fbde456163cdb37 2023-12-10 07:00:56,996 INFO [StoreOpener-738cc1ffb58c4f3e8fbde456163cdb37-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf of region 738cc1ffb58c4f3e8fbde456163cdb37 2023-12-10 07:00:56,997 DEBUG [StoreOpener-738cc1ffb58c4f3e8fbde456163cdb37-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin0/738cc1ffb58c4f3e8fbde456163cdb37/cf 2023-12-10 07:00:56,997 DEBUG [StoreOpener-738cc1ffb58c4f3e8fbde456163cdb37-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin0/738cc1ffb58c4f3e8fbde456163cdb37/cf 2023-12-10 07:00:56,999 INFO [StoreOpener-738cc1ffb58c4f3e8fbde456163cdb37-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:10, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 738cc1ffb58c4f3e8fbde456163cdb37 columnFamilyName cf 2023-12-10 07:00:57,000 INFO [StoreOpener-738cc1ffb58c4f3e8fbde456163cdb37-1] regionserver.HStore(310): Store=738cc1ffb58c4f3e8fbde456163cdb37/cf, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-12-10 07:00:57,001 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin0/738cc1ffb58c4f3e8fbde456163cdb37 2023-12-10 07:00:57,002 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin0/738cc1ffb58c4f3e8fbde456163cdb37 2023-12-10 07:00:57,006 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 738cc1ffb58c4f3e8fbde456163cdb37 2023-12-10 07:00:57,007 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 738cc1ffb58c4f3e8fbde456163cdb37; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71577194, jitterRate=0.0665833055973053}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-12-10 07:00:57,007 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 738cc1ffb58c4f3e8fbde456163cdb37: 2023-12-10 07:00:57,007 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37., pid=29, masterSystemTime=1705291897373 2023-12-10 07:00:57,009 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37. 2023-12-10 07:00:57,009 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37. 2023-12-10 07:00:57,010 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=27 updating hbase:meta row=738cc1ffb58c4f3e8fbde456163cdb37, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,42073,1702191353745 2023-12-10 07:00:57,010 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1705291897873"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1705291897873"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1705291897873"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1705291897873"}]},"ts":"1705291897873"} 2023-12-10 07:00:57,013 INFO [PEWorker-3] procedure2.ProcedureExecutor(1953): Finished subprocedure pid=29, resume processing ppid=27 2023-12-10 07:00:57,014 INFO [PEWorker-3] procedure2.ProcedureExecutor(1457): Finished pid=29, ppid=27, state=SUCCESS; OpenRegionProcedure 738cc1ffb58c4f3e8fbde456163cdb37, server=jenkins-hbase4.apache.org,42073,1702191353745 in 1.8000 sec 2023-12-10 07:00:57,015 INFO [PEWorker-4] procedure2.ProcedureExecutor(1457): Finished pid=27, state=SUCCESS; TransitRegionStateProcedure table=TestQuotaAdmin0, region=738cc1ffb58c4f3e8fbde456163cdb37, REOPEN/MOVE in 5.4000 sec 2023-12-10 07:00:57,098 INFO [master/jenkins-hbase4:0.Chore.1] master.HMaster(1857): balance hri=700b1850706b553c6f995417c6bb917e, source=jenkins-hbase4.apache.org,37997,1702191353702, destination=jenkins-hbase4.apache.org,42073,1702191353745 2023-12-10 07:00:57,099 DEBUG [master/jenkins-hbase4:0.Chore.1] procedure2.ProcedureExecutor(1075): Stored pid=30, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=TestQuotaAdmin2, region=700b1850706b553c6f995417c6bb917e, REOPEN/MOVE 2023-12-10 07:00:57,100 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=TestQuotaAdmin2, region=700b1850706b553c6f995417c6bb917e, REOPEN/MOVE 2023-12-10 07:00:57,101 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=700b1850706b553c6f995417c6bb917e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 07:00:57,101 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1705291898573"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1705291898573"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1705291898573"}]},"ts":"1705291898573"} 2023-12-10 07:00:57,102 INFO [PEWorker-5] procedure2.ProcedureExecutor(1796): Initialized subprocedures=[{pid=31, ppid=30, state=RUNNABLE; CloseRegionProcedure 700b1850706b553c6f995417c6bb917e, server=jenkins-hbase4.apache.org,37997,1702191353702}] 2023-12-10 07:00:57,254 DEBUG [RSProcedureDispatcher-pool-5] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 07:00:57,255 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 700b1850706b553c6f995417c6bb917e 2023-12-10 07:00:57,255 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 700b1850706b553c6f995417c6bb917e, disabling compactions & flushes 2023-12-10 07:00:57,255 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e. 2023-12-10 07:00:57,255 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e. 2023-12-10 07:00:57,255 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e. after waiting 0 ms 2023-12-10 07:00:57,255 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e. 2023-12-10 07:00:57,260 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin2/700b1850706b553c6f995417c6bb917e/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-12-10 07:00:57,262 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e. 2023-12-10 07:00:57,262 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 700b1850706b553c6f995417c6bb917e: 2023-12-10 07:00:57,262 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3515): Adding 700b1850706b553c6f995417c6bb917e move to jenkins-hbase4.apache.org,42073,1702191353745 record at close sequenceid=2 2023-12-10 07:00:57,263 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 700b1850706b553c6f995417c6bb917e 2023-12-10 07:00:57,264 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=700b1850706b553c6f995417c6bb917e, regionState=CLOSED 2023-12-10 07:00:57,264 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1705291900073"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1705291900073"}]},"ts":"1705291900073"} 2023-12-10 07:00:57,268 INFO [PEWorker-2] procedure2.ProcedureExecutor(1953): Finished subprocedure pid=31, resume processing ppid=30 2023-12-10 07:00:57,268 INFO [PEWorker-2] procedure2.ProcedureExecutor(1457): Finished pid=31, ppid=30, state=SUCCESS; CloseRegionProcedure 700b1850706b553c6f995417c6bb917e, server=jenkins-hbase4.apache.org,37997,1702191353702 in 1.5000 sec 2023-12-10 07:00:57,269 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=30, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=TestQuotaAdmin2, region=700b1850706b553c6f995417c6bb917e, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,42073,1702191353745; forceNewPlan=false, retain=false 2023-12-10 07:00:57,419 INFO [jenkins-hbase4:40533] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-12-10 07:00:57,420 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=700b1850706b553c6f995417c6bb917e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42073,1702191353745 2023-12-10 07:00:57,420 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1705291900873"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1705291900873"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1705291900873"}]},"ts":"1705291900873"} 2023-12-10 07:00:57,422 INFO [PEWorker-4] procedure2.ProcedureExecutor(1796): Initialized subprocedures=[{pid=32, ppid=30, state=RUNNABLE; OpenRegionProcedure 700b1850706b553c6f995417c6bb917e, server=jenkins-hbase4.apache.org,42073,1702191353745}] 2023-12-10 07:00:57,574 DEBUG [RSProcedureDispatcher-pool-3] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,42073,1702191353745 2023-12-10 07:00:57,577 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e. 2023-12-10 07:00:57,578 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 700b1850706b553c6f995417c6bb917e, NAME => 'TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e.', STARTKEY => '', ENDKEY => ''} 2023-12-10 07:00:57,578 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table TestQuotaAdmin2 700b1850706b553c6f995417c6bb917e 2023-12-10 07:00:57,578 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-12-10 07:00:57,578 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 700b1850706b553c6f995417c6bb917e 2023-12-10 07:00:57,578 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 700b1850706b553c6f995417c6bb917e 2023-12-10 07:00:57,579 INFO [StoreOpener-700b1850706b553c6f995417c6bb917e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf of region 700b1850706b553c6f995417c6bb917e 2023-12-10 07:00:57,581 DEBUG [StoreOpener-700b1850706b553c6f995417c6bb917e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin2/700b1850706b553c6f995417c6bb917e/cf 2023-12-10 07:00:57,581 DEBUG [StoreOpener-700b1850706b553c6f995417c6bb917e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin2/700b1850706b553c6f995417c6bb917e/cf 2023-12-10 07:00:57,581 INFO [StoreOpener-700b1850706b553c6f995417c6bb917e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:10, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 700b1850706b553c6f995417c6bb917e columnFamilyName cf 2023-12-10 07:00:57,582 INFO [StoreOpener-700b1850706b553c6f995417c6bb917e-1] regionserver.HStore(310): Store=700b1850706b553c6f995417c6bb917e/cf, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-12-10 07:00:57,583 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin2/700b1850706b553c6f995417c6bb917e 2023-12-10 07:00:57,585 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin2/700b1850706b553c6f995417c6bb917e 2023-12-10 07:00:57,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 700b1850706b553c6f995417c6bb917e 2023-12-10 07:00:57,589 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 700b1850706b553c6f995417c6bb917e; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72958293, jitterRate=0.0871632844209671}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-12-10 07:00:57,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 700b1850706b553c6f995417c6bb917e: 2023-12-10 07:00:57,590 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e., pid=32, masterSystemTime=1705291902373 2023-12-10 07:00:57,592 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e. 2023-12-10 07:00:57,592 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e. 2023-12-10 07:00:57,593 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=700b1850706b553c6f995417c6bb917e, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,42073,1702191353745 2023-12-10 07:00:57,593 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1705291902573"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1705291902573"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1705291902573"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1705291902573"}]},"ts":"1705291902573"} 2023-12-10 07:00:57,600 INFO [PEWorker-1] procedure2.ProcedureExecutor(1953): Finished subprocedure pid=32, resume processing ppid=30 2023-12-10 07:00:57,600 INFO [PEWorker-1] procedure2.ProcedureExecutor(1457): Finished pid=32, ppid=30, state=SUCCESS; OpenRegionProcedure 700b1850706b553c6f995417c6bb917e, server=jenkins-hbase4.apache.org,42073,1702191353745 in 1.9000 sec 2023-12-10 07:00:57,602 INFO [PEWorker-2] procedure2.ProcedureExecutor(1457): Finished pid=30, state=SUCCESS; TransitRegionStateProcedure table=TestQuotaAdmin2, region=700b1850706b553c6f995417c6bb917e, REOPEN/MOVE in 4.4000 sec 2023-12-10 07:00:57,699 INFO [master/jenkins-hbase4:0.Chore.1] master.HMaster(1857): balance hri=dca2e6ecbeef49fb8929fd36dd74df8b, source=jenkins-hbase4.apache.org,37997,1702191353702, destination=jenkins-hbase4.apache.org,42073,1702191353745 2023-12-10 07:00:57,701 DEBUG [master/jenkins-hbase4:0.Chore.1] procedure2.ProcedureExecutor(1075): Stored pid=33, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=hbase:quota, region=dca2e6ecbeef49fb8929fd36dd74df8b, REOPEN/MOVE 2023-12-10 07:00:57,702 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=33, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=hbase:quota, region=dca2e6ecbeef49fb8929fd36dd74df8b, REOPEN/MOVE 2023-12-10 07:00:57,703 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=33 updating hbase:meta row=dca2e6ecbeef49fb8929fd36dd74df8b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 07:00:57,703 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b.","families":{"info":[{"qualifier":"regioninfo","vlen":37,"tag":[],"timestamp":"1705291903673"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1705291903673"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1705291903673"}]},"ts":"1705291903673"} 2023-12-10 07:00:57,705 INFO [PEWorker-3] procedure2.ProcedureExecutor(1796): Initialized subprocedures=[{pid=34, ppid=33, state=RUNNABLE; CloseRegionProcedure dca2e6ecbeef49fb8929fd36dd74df8b, server=jenkins-hbase4.apache.org,37997,1702191353702}] 2023-12-10 07:00:57,857 DEBUG [RSProcedureDispatcher-pool-4] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 07:00:57,857 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close dca2e6ecbeef49fb8929fd36dd74df8b 2023-12-10 07:00:57,857 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing dca2e6ecbeef49fb8929fd36dd74df8b, disabling compactions & flushes 2023-12-10 07:00:57,857 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b. 2023-12-10 07:00:57,857 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b. 2023-12-10 07:00:57,857 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b. after waiting 0 ms 2023-12-10 07:00:57,857 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b. 2023-12-10 07:00:57,866 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/recovered.edits/17.seqid, newMaxSeqId=17, maxSeqId=1 2023-12-10 07:00:57,867 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b. 2023-12-10 07:00:57,867 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for dca2e6ecbeef49fb8929fd36dd74df8b: 2023-12-10 07:00:57,867 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3515): Adding dca2e6ecbeef49fb8929fd36dd74df8b move to jenkins-hbase4.apache.org,42073,1702191353745 record at close sequenceid=14 2023-12-10 07:00:57,869 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed dca2e6ecbeef49fb8929fd36dd74df8b 2023-12-10 07:00:57,870 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=33 updating hbase:meta row=dca2e6ecbeef49fb8929fd36dd74df8b, regionState=CLOSED 2023-12-10 07:00:57,870 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b.","families":{"info":[{"qualifier":"regioninfo","vlen":37,"tag":[],"timestamp":"1705291906573"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1705291906573"}]},"ts":"1705291906573"} 2023-12-10 07:00:57,874 INFO [PEWorker-5] procedure2.ProcedureExecutor(1953): Finished subprocedure pid=34, resume processing ppid=33 2023-12-10 07:00:57,874 INFO [PEWorker-5] procedure2.ProcedureExecutor(1457): Finished pid=34, ppid=33, state=SUCCESS; CloseRegionProcedure dca2e6ecbeef49fb8929fd36dd74df8b, server=jenkins-hbase4.apache.org,37997,1702191353702 in 2.9000 sec 2023-12-10 07:00:57,875 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=33, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:quota, region=dca2e6ecbeef49fb8929fd36dd74df8b, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,42073,1702191353745; forceNewPlan=false, retain=false 2023-12-10 07:00:58,025 INFO [jenkins-hbase4:40533] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-12-10 07:00:58,025 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=33 updating hbase:meta row=dca2e6ecbeef49fb8929fd36dd74df8b, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,42073,1702191353745 2023-12-10 07:00:58,025 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b.","families":{"info":[{"qualifier":"regioninfo","vlen":37,"tag":[],"timestamp":"1705291907573"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1705291907573"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1705291907573"}]},"ts":"1705291907573"} 2023-12-10 07:00:58,027 INFO [PEWorker-2] procedure2.ProcedureExecutor(1796): Initialized subprocedures=[{pid=35, ppid=33, state=RUNNABLE; OpenRegionProcedure dca2e6ecbeef49fb8929fd36dd74df8b, server=jenkins-hbase4.apache.org,42073,1702191353745}] 2023-12-10 07:00:58,179 DEBUG [RSProcedureDispatcher-pool-5] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,42073,1702191353745 2023-12-10 07:00:58,183 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b. 2023-12-10 07:00:58,183 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => dca2e6ecbeef49fb8929fd36dd74df8b, NAME => 'hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b.', STARTKEY => '', ENDKEY => ''} 2023-12-10 07:00:58,184 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table quota dca2e6ecbeef49fb8929fd36dd74df8b 2023-12-10 07:00:58,184 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-12-10 07:00:58,184 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for dca2e6ecbeef49fb8929fd36dd74df8b 2023-12-10 07:00:58,184 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for dca2e6ecbeef49fb8929fd36dd74df8b 2023-12-10 07:00:58,185 INFO [StoreOpener-dca2e6ecbeef49fb8929fd36dd74df8b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family q of region dca2e6ecbeef49fb8929fd36dd74df8b 2023-12-10 07:00:58,186 DEBUG [StoreOpener-dca2e6ecbeef49fb8929fd36dd74df8b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/q 2023-12-10 07:00:58,186 DEBUG [StoreOpener-dca2e6ecbeef49fb8929fd36dd74df8b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/q 2023-12-10 07:00:58,187 INFO [StoreOpener-dca2e6ecbeef49fb8929fd36dd74df8b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:10, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dca2e6ecbeef49fb8929fd36dd74df8b columnFamilyName q 2023-12-10 07:00:58,201 DEBUG [StoreOpener-dca2e6ecbeef49fb8929fd36dd74df8b-1] regionserver.HStore(539): loaded hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/q/01f660fa9b55470f95c556f0b2cb2707 2023-12-10 07:00:58,206 INFO [StoreFileOpener-q-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for a0e8d98621be4825946bd6a3dee2f5e8 2023-12-10 07:00:58,207 DEBUG [StoreOpener-dca2e6ecbeef49fb8929fd36dd74df8b-1] regionserver.HStore(539): loaded hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/q/a0e8d98621be4825946bd6a3dee2f5e8 2023-12-10 07:00:58,207 INFO [StoreOpener-dca2e6ecbeef49fb8929fd36dd74df8b-1] regionserver.HStore(310): Store=dca2e6ecbeef49fb8929fd36dd74df8b/q, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-12-10 07:00:58,207 INFO [StoreOpener-dca2e6ecbeef49fb8929fd36dd74df8b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family u of region dca2e6ecbeef49fb8929fd36dd74df8b 2023-12-10 07:00:58,208 DEBUG [StoreOpener-dca2e6ecbeef49fb8929fd36dd74df8b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/u 2023-12-10 07:00:58,208 DEBUG [StoreOpener-dca2e6ecbeef49fb8929fd36dd74df8b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/u 2023-12-10 07:00:58,209 INFO [StoreOpener-dca2e6ecbeef49fb8929fd36dd74df8b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:10, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dca2e6ecbeef49fb8929fd36dd74df8b columnFamilyName u 2023-12-10 07:00:58,219 INFO [StoreFileOpener-u-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dd54a76871ef426db9514b247981de5a 2023-12-10 07:00:58,219 DEBUG [StoreOpener-dca2e6ecbeef49fb8929fd36dd74df8b-1] regionserver.HStore(539): loaded hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/u/dd54a76871ef426db9514b247981de5a 2023-12-10 07:00:58,219 INFO [StoreOpener-dca2e6ecbeef49fb8929fd36dd74df8b-1] regionserver.HStore(310): Store=dca2e6ecbeef49fb8929fd36dd74df8b/u, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-12-10 07:00:58,220 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b 2023-12-10 07:00:58,224 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b 2023-12-10 07:00:58,226 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:quota descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-12-10 07:00:58,228 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for dca2e6ecbeef49fb8929fd36dd74df8b 2023-12-10 07:00:58,229 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened dca2e6ecbeef49fb8929fd36dd74df8b; next sequenceid=18; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69722275, jitterRate=0.0389428585767746}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-12-10 07:00:58,229 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for dca2e6ecbeef49fb8929fd36dd74df8b: 2023-12-10 07:00:58,230 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2339): Post open deploy tasks for hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., pid=35, masterSystemTime=1705291909873 2023-12-10 07:00:58,231 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2366): Finished post open deploy task for hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b. 2023-12-10 07:00:58,231 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b. 2023-12-10 07:00:58,232 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=33 updating hbase:meta row=dca2e6ecbeef49fb8929fd36dd74df8b, regionState=OPEN, openSeqNum=18, regionLocation=jenkins-hbase4.apache.org,42073,1702191353745 2023-12-10 07:00:58,232 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b.","families":{"info":[{"qualifier":"regioninfo","vlen":37,"tag":[],"timestamp":"1705291944173"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1705291944173"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1705291944173"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1705291944173"}]},"ts":"1705291944173"} 2023-12-10 07:00:58,236 INFO [PEWorker-4] procedure2.ProcedureExecutor(1953): Finished subprocedure pid=35, resume processing ppid=33 2023-12-10 07:00:58,236 INFO [PEWorker-4] procedure2.ProcedureExecutor(1457): Finished pid=35, ppid=33, state=SUCCESS; OpenRegionProcedure dca2e6ecbeef49fb8929fd36dd74df8b, server=jenkins-hbase4.apache.org,42073,1702191353745 in 36.5000 sec 2023-12-10 07:00:58,238 INFO [PEWorker-5] procedure2.ProcedureExecutor(1457): Finished pid=33, state=SUCCESS; TransitRegionStateProcedure table=hbase:quota, region=dca2e6ecbeef49fb8929fd36dd74df8b, REOPEN/MOVE in 41.0000 sec 2023-12-10 07:00:58,301 DEBUG [master/jenkins-hbase4:0.Chore.1] master.HMaster(1888): Balancer is going into sleep until next period in 300000ms 2023-12-10 07:01:00,893 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-12-10 07:01:07,127 INFO [regionserver/jenkins-hbase4:0.Chore.3] hbase.ScheduledChore(142): Chore: RegionSizeReportingChore missed its start time 2023-12-10 07:01:07,774 INFO [regionserver/jenkins-hbase4:0.Chore.3] regionserver.HRegionServer$PeriodicMemStoreFlusher(1921): MemstoreFlusherChore requesting flush of hbase:meta,,1.1588230740 because 1588230740/info has an old edit so flush to free WALs after random delay 77619 ms 2023-12-10 07:01:10,077 DEBUG [MemStoreFlusher.0] regionserver.FlushAllLargeStoresPolicy(69): Since none of the CFs were above the size, flushing all. 2023-12-10 07:01:10,078 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=4.60 KB heapSize=8.23 KB 2023-12-10 07:01:10,094 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=4.60 KB at sequenceid=51 (bloomFilter=false), to=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740/.tmp/info/029f3603afce4cc78f38fd2c27274e40 2023-12-10 07:01:10,102 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740/.tmp/info/029f3603afce4cc78f38fd2c27274e40 as hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740/info/029f3603afce4cc78f38fd2c27274e40 2023-12-10 07:01:10,110 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740/info/029f3603afce4cc78f38fd2c27274e40, entries=33, sequenceid=51, filesize=8.6 K 2023-12-10 07:01:10,111 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~4.60 KB/4706, heapSize ~7.71 KB/7896, currentSize=0 B/0 for 1588230740 in 1000ms, sequenceid=51, compaction requested=false 2023-12-10 07:01:10,111 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 1588230740: 2023-12-10 07:01:10,236 DEBUG [hconnection-0x60f71901-shared-pool-16] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-12-10 07:01:10,238 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41370, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=ClientService 2023-12-10 07:01:12,534 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.CallRunner(144): callId: 94 service: ClientService methodName: Scan size: 139 connection: 172.31.14.131:41678 deadline: 1705292449873, exception=org.apache.hadoop.hbase.exceptions.RegionMovedException: Region moved to: hostname=jenkins-hbase4.apache.org port=42073 startCode=1702191353745. As of locationSeqNum=14. 2023-12-10 07:01:12,789 DEBUG [hconnection-0x2ae805ba-shared-pool-12] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-12-10 07:01:12,790 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49004, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-12-10 07:01:54,741 INFO [regionserver/jenkins-hbase4:0.Chore.3] hbase.ScheduledChore(142): Chore: SpaceQuotaRefresherChore missed its start time 2023-12-10 07:01:54,742 INFO [regionserver/jenkins-hbase4:0.Chore.8] hbase.ScheduledChore(142): Chore: CompactionChecker missed its start time 2023-12-10 07:01:54,742 INFO [regionserver/jenkins-hbase4:0.Chore.2] hbase.ScheduledChore(142): Chore: MemstoreFlusherChore missed its start time 2023-12-10 07:01:54,817 DEBUG [regionserver/jenkins-hbase4:0.Chore.5] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-12-10 07:01:54,820 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54530, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=MasterService 2023-12-10 07:01:54,825 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":2,"starttimems":"1705293687973","responsesize":"1025","method":"Multi","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., for 2 action(s) and 1st row key=n.default","processingtimems":55800,"client":"172.31.14.131:35514","multi.service_calls":0,"queuetimems":1300,"class":"MiniHBaseClusterRegionServer","multi.mutations":0} 2023-12-10 07:01:54,825 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Multi(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$MultiRequest)","multi.gets":2,"starttimems":"1705293689473","responsesize":"1025","method":"Multi","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., for 2 action(s) and 1st row key=n.default","processingtimems":54300,"client":"172.31.14.131:35514","multi.service_calls":0,"queuetimems":500,"class":"MiniHBaseClusterRegionServer","multi.mutations":0} 2023-12-10 07:01:54,825 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.CallRunner(105): Dropping timed out call: callId: 8475 service: ClientService methodName: Multi size: 174 connection: 172.31.14.131:35514 deadline: 1705293749673 param: region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., for 2 action(s) and 1st row key=n.default connection: 172.31.14.131:35514 2023-12-10 07:01:54,832 INFO [regionserver/jenkins-hbase4:0.Chore.8] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,832 INFO [regionserver/jenkins-hbase4:0.Chore.2] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,832 INFO [regionserver/jenkins-hbase4:0.Chore.5] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,833 INFO [regionserver/jenkins-hbase4:0.Chore.1] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,833 INFO [regionserver/jenkins-hbase4:0.Chore.7] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,834 INFO [regionserver/jenkins-hbase4:0.Chore.3] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,834 INFO [regionserver/jenkins-hbase4:0.Chore.4] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,834 INFO [regionserver/jenkins-hbase4:0.Chore.6] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,843 INFO [regionserver/jenkins-hbase4:0.Chore.8] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,843 INFO [regionserver/jenkins-hbase4:0.Chore.2] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,843 INFO [regionserver/jenkins-hbase4:0.Chore.5] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,843 INFO [regionserver/jenkins-hbase4:0.Chore.1] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,843 INFO [regionserver/jenkins-hbase4:0.Chore.7] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,844 INFO [regionserver/jenkins-hbase4:0.Chore.3] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,844 INFO [regionserver/jenkins-hbase4:0.Chore.4] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,844 INFO [regionserver/jenkins-hbase4:0.Chore.6] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,853 INFO [regionserver/jenkins-hbase4:0.Chore.8] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,853 INFO [regionserver/jenkins-hbase4:0.Chore.2] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,853 INFO [regionserver/jenkins-hbase4:0.Chore.5] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,853 INFO [regionserver/jenkins-hbase4:0.Chore.1] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,853 INFO [regionserver/jenkins-hbase4:0.Chore.7] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,854 INFO [regionserver/jenkins-hbase4:0.Chore.3] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,854 INFO [regionserver/jenkins-hbase4:0.Chore.4] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,854 INFO [regionserver/jenkins-hbase4:0.Chore.6] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,863 INFO [regionserver/jenkins-hbase4:0.Chore.8] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,863 INFO [regionserver/jenkins-hbase4:0.Chore.2] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,863 INFO [regionserver/jenkins-hbase4:0.Chore.5] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,863 INFO [regionserver/jenkins-hbase4:0.Chore.1] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,863 INFO [regionserver/jenkins-hbase4:0.Chore.7] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,864 INFO [regionserver/jenkins-hbase4:0.Chore.3] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,864 INFO [regionserver/jenkins-hbase4:0.Chore.4] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,865 INFO [regionserver/jenkins-hbase4:0.Chore.6] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,873 INFO [regionserver/jenkins-hbase4:0.Chore.8] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,873 INFO [regionserver/jenkins-hbase4:0.Chore.2] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,873 INFO [regionserver/jenkins-hbase4:0.Chore.5] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,873 INFO [regionserver/jenkins-hbase4:0.Chore.1] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,873 INFO [regionserver/jenkins-hbase4:0.Chore.7] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,874 INFO [regionserver/jenkins-hbase4:0.Chore.3] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,874 INFO [regionserver/jenkins-hbase4:0.Chore.4] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,875 INFO [regionserver/jenkins-hbase4:0.Chore.6] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,883 INFO [regionserver/jenkins-hbase4:0.Chore.8] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,883 INFO [regionserver/jenkins-hbase4:0.Chore.2] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,884 INFO [regionserver/jenkins-hbase4:0.Chore.5] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,884 INFO [regionserver/jenkins-hbase4:0.Chore.1] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,884 INFO [regionserver/jenkins-hbase4:0.Chore.7] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,884 INFO [regionserver/jenkins-hbase4:0.Chore.3] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,885 INFO [regionserver/jenkins-hbase4:0.Chore.4] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:54,885 INFO [regionserver/jenkins-hbase4:0.Chore.6] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,000 INFO [regionserver/jenkins-hbase4:0.Chore.1] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,000 INFO [regionserver/jenkins-hbase4:0.Chore.7] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,000 INFO [regionserver/jenkins-hbase4:0.Chore.2] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,000 INFO [regionserver/jenkins-hbase4:0.Chore.3] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,000 INFO [regionserver/jenkins-hbase4:0.Chore.8] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,000 INFO [regionserver/jenkins-hbase4:0.Chore.5] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,001 DEBUG [regionserver/jenkins-hbase4:0.Chore.3] quotas.FileSystemUtilizationChore(99): Preempting execution of FileSystemUtilizationChore because it exceeds the maximum iteration configuration value. Will process remaining Regions on a subsequent invocation. 2023-12-10 07:01:55,010 INFO [regionserver/jenkins-hbase4:0.Chore.4] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,011 INFO [regionserver/jenkins-hbase4:0.Chore.6] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,011 INFO [regionserver/jenkins-hbase4:0.Chore.1] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,011 INFO [regionserver/jenkins-hbase4:0.Chore.7] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,011 INFO [regionserver/jenkins-hbase4:0.Chore.2] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,011 INFO [regionserver/jenkins-hbase4:0.Chore.3] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,011 INFO [regionserver/jenkins-hbase4:0.Chore.8] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,011 INFO [regionserver/jenkins-hbase4:0.Chore.5] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,020 INFO [regionserver/jenkins-hbase4:0.Chore.4] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,021 INFO [regionserver/jenkins-hbase4:0.Chore.6] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,021 INFO [regionserver/jenkins-hbase4:0.Chore.1] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,021 INFO [regionserver/jenkins-hbase4:0.Chore.7] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,021 INFO [regionserver/jenkins-hbase4:0.Chore.2] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,021 INFO [regionserver/jenkins-hbase4:0.Chore.3] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,021 INFO [regionserver/jenkins-hbase4:0.Chore.8] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,021 INFO [regionserver/jenkins-hbase4:0.Chore.5] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,030 INFO [regionserver/jenkins-hbase4:0.Chore.4] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,031 INFO [regionserver/jenkins-hbase4:0.Chore.6] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,031 INFO [regionserver/jenkins-hbase4:0.Chore.1] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,031 INFO [regionserver/jenkins-hbase4:0.Chore.7] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,031 INFO [regionserver/jenkins-hbase4:0.Chore.2] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,031 INFO [regionserver/jenkins-hbase4:0.Chore.3] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,031 INFO [regionserver/jenkins-hbase4:0.Chore.8] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,031 INFO [regionserver/jenkins-hbase4:0.Chore.5] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,040 INFO [regionserver/jenkins-hbase4:0.Chore.4] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,041 INFO [regionserver/jenkins-hbase4:0.Chore.6] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,041 INFO [regionserver/jenkins-hbase4:0.Chore.1] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,041 INFO [regionserver/jenkins-hbase4:0.Chore.7] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,041 INFO [regionserver/jenkins-hbase4:0.Chore.2] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,041 INFO [regionserver/jenkins-hbase4:0.Chore.3] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,042 INFO [regionserver/jenkins-hbase4:0.Chore.8] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,042 INFO [regionserver/jenkins-hbase4:0.Chore.5] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,050 INFO [regionserver/jenkins-hbase4:0.Chore.4] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,051 INFO [regionserver/jenkins-hbase4:0.Chore.6] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,051 INFO [regionserver/jenkins-hbase4:0.Chore.1] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,051 INFO [regionserver/jenkins-hbase4:0.Chore.7] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,052 INFO [regionserver/jenkins-hbase4:0.Chore.2] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,052 INFO [regionserver/jenkins-hbase4:0.Chore.3] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,052 INFO [regionserver/jenkins-hbase4:0.Chore.8] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,052 INFO [regionserver/jenkins-hbase4:0.Chore.5] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,060 INFO [regionserver/jenkins-hbase4:0.Chore.4] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,061 INFO [regionserver/jenkins-hbase4:0.Chore.6] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,061 INFO [regionserver/jenkins-hbase4:0.Chore.1] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,061 INFO [regionserver/jenkins-hbase4:0.Chore.7] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,062 INFO [regionserver/jenkins-hbase4:0.Chore.2] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,062 INFO [regionserver/jenkins-hbase4:0.Chore.3] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,062 INFO [regionserver/jenkins-hbase4:0.Chore.8] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,062 INFO [regionserver/jenkins-hbase4:0.Chore.5] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,071 INFO [regionserver/jenkins-hbase4:0.Chore.4] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,071 INFO [regionserver/jenkins-hbase4:0.Chore.6] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,071 INFO [regionserver/jenkins-hbase4:0.Chore.1] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,071 INFO [regionserver/jenkins-hbase4:0.Chore.7] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,072 INFO [regionserver/jenkins-hbase4:0.Chore.2] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,072 INFO [regionserver/jenkins-hbase4:0.Chore.3] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,072 INFO [regionserver/jenkins-hbase4:0.Chore.8] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,072 INFO [regionserver/jenkins-hbase4:0.Chore.5] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,081 INFO [regionserver/jenkins-hbase4:0.Chore.4] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,081 INFO [regionserver/jenkins-hbase4:0.Chore.6] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,082 INFO [regionserver/jenkins-hbase4:0.Chore.1] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,082 INFO [regionserver/jenkins-hbase4:0.Chore.7] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,082 INFO [regionserver/jenkins-hbase4:0.Chore.2] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,082 INFO [regionserver/jenkins-hbase4:0.Chore.3] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,082 INFO [regionserver/jenkins-hbase4:0.Chore.8] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,082 INFO [regionserver/jenkins-hbase4:0.Chore.5] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,091 INFO [regionserver/jenkins-hbase4:0.Chore.4] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,091 INFO [regionserver/jenkins-hbase4:0.Chore.6] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,092 INFO [regionserver/jenkins-hbase4:0.Chore.1] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,092 INFO [regionserver/jenkins-hbase4:0.Chore.7] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,092 INFO [regionserver/jenkins-hbase4:0.Chore.2] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,092 INFO [regionserver/jenkins-hbase4:0.Chore.3] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,092 INFO [regionserver/jenkins-hbase4:0.Chore.8] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,092 INFO [regionserver/jenkins-hbase4:0.Chore.5] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,101 INFO [regionserver/jenkins-hbase4:0.Chore.4] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,102 INFO [regionserver/jenkins-hbase4:0.Chore.6] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,102 INFO [regionserver/jenkins-hbase4:0.Chore.1] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,102 INFO [regionserver/jenkins-hbase4:0.Chore.7] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,102 INFO [regionserver/jenkins-hbase4:0.Chore.2] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,102 INFO [regionserver/jenkins-hbase4:0.Chore.3] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,102 INFO [regionserver/jenkins-hbase4:0.Chore.8] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,102 INFO [regionserver/jenkins-hbase4:0.Chore.5] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,103 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Scan(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ScanRequest)","starttimems":"1705300049573","responsesize":"24","method":"Scan","param":"region { type: REGION_NAME value: \"hbase:meta,,1\" } scan { column { family: \"info\" } start_row: \"hbase:quota,n.default,99999999999999\" stop_row: \"hbas ","processingtimems":98500,"client":"172.31.14.131:35514","queuetimems":700,"class":"MiniHBaseClusterRegionServer","scandetails":"table: hbase:meta region: hbase:meta,,1.1588230740"} 2023-12-10 07:01:55,105 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37997] ipc.RpcServer(528): (responseTooSlow): {"call":"Scan(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ScanRequest)","starttimems":"1705300187273","responsesize":"737","method":"Scan","param":"scanner_id: 8252208067836379179 number_of_rows: 1 close_scanner: false next_call_seq: 0 client_handles_partials: true client_handles_heartbeats: true ","processingtimems":45200,"client":"172.31.14.131:35514","queuetimems":500,"class":"MiniHBaseClusterRegionServer","scandetails":"table: hbase:meta region: hbase:meta,,1.1588230740"} 2023-12-10 07:01:55,107 WARN [hconnection-0x214824e3-shared-pool-11] client.AsyncRequestFutureImpl(819): id=2, table=hbase:quota, attempt=2/46, failureCount=2ops, last exception=org.apache.hadoop.hbase.client.OperationTimeoutExceededException: Timeout exceeded before call began. Meta requests may be slow, the operation timeout is too short for the number of requests, or the configured retries can't complete in the operation timeout. on jenkins-hbase4.apache.org,42073,1702191353745, tracking started Sun Dec 10 07:01:54 UTC 2023; NOT retrying, failed=2 -- final attempt! 2023-12-10 07:01:55,107 WARN [hconnection-0x214824e3-shared-pool-9] client.AsyncRequestFutureImpl(819): id=2, table=hbase:quota, attempt=2/46, failureCount=2ops, last exception=org.apache.hadoop.hbase.client.OperationTimeoutExceededException: Timeout exceeded before call began. Meta requests may be slow, the operation timeout is too short for the number of requests, or the configured retries can't complete in the operation timeout. on jenkins-hbase4.apache.org,42073,1702191353745, tracking started Sun Dec 10 07:01:54 UTC 2023; NOT retrying, failed=2 -- final attempt! 2023-12-10 07:01:55,107 WARN [hconnection-0x214824e3-shared-pool-7] client.AsyncRequestFutureImpl(819): id=2, table=hbase:quota, attempt=2/46, failureCount=2ops, last exception=org.apache.hadoop.hbase.client.OperationTimeoutExceededException: Timeout exceeded before call began. Meta requests may be slow, the operation timeout is too short for the number of requests, or the configured retries can't complete in the operation timeout. on jenkins-hbase4.apache.org,42073,1702191353745, tracking started Sun Dec 10 07:01:54 UTC 2023; NOT retrying, failed=2 -- final attempt! 2023-12-10 07:01:55,107 WARN [hconnection-0x214824e3-shared-pool-8] client.AsyncRequestFutureImpl(819): id=2, table=hbase:quota, attempt=2/46, failureCount=2ops, last exception=org.apache.hadoop.hbase.client.OperationTimeoutExceededException: Timeout exceeded before call began. Meta requests may be slow, the operation timeout is too short for the number of requests, or the configured retries can't complete in the operation timeout. on jenkins-hbase4.apache.org,42073,1702191353745, tracking started Sun Dec 10 07:01:54 UTC 2023; NOT retrying, failed=2 -- final attempt! 2023-12-10 07:01:55,110 WARN [regionserver/jenkins-hbase4:0.Chore.6] quotas.QuotaCache$QuotaRefresherChore(344): Unable to read namespace from quota table org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 2 actions: Timeout exceeded before call began. Meta requests may be slow, the operation timeout is too short for the number of requests, or the configured retries can't complete in the operation timeout.: 2 times, servers with issues: jenkins-hbase4.apache.org,42073,1702191353745 at org.apache.hadoop.hbase.client.BatchErrors.makeException(BatchErrors.java:50) at org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.getErrors(AsyncRequestFutureImpl.java:1236) at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:434) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:389) at org.apache.hadoop.hbase.quotas.QuotaTableUtil.doGet(QuotaTableUtil.java:910) at org.apache.hadoop.hbase.quotas.QuotaUtil.fetchGlobalQuotas(QuotaUtil.java:374) at org.apache.hadoop.hbase.quotas.QuotaUtil.fetchNamespaceQuotas(QuotaUtil.java:341) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore$1.fetchEntries(QuotaCache.java:242) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetch(QuotaCache.java:333) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetchNamespaceQuotaState(QuotaCache.java:234) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.chore(QuotaCache.java:225) at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:107) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-12-10 07:01:55,107 WARN [hconnection-0x214824e3-shared-pool-12] client.AsyncRequestFutureImpl(819): id=2, table=hbase:quota, attempt=2/46, failureCount=2ops, last exception=org.apache.hadoop.hbase.client.OperationTimeoutExceededException: Timeout exceeded before call began. Meta requests may be slow, the operation timeout is too short for the number of requests, or the configured retries can't complete in the operation timeout. on jenkins-hbase4.apache.org,42073,1702191353745, tracking started Sun Dec 10 07:01:54 UTC 2023; NOT retrying, failed=2 -- final attempt! 2023-12-10 07:01:55,110 WARN [regionserver/jenkins-hbase4:0.Chore.3] quotas.QuotaCache$QuotaRefresherChore(344): Unable to read namespace from quota table org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 2 actions: Timeout exceeded before call began. Meta requests may be slow, the operation timeout is too short for the number of requests, or the configured retries can't complete in the operation timeout.: 2 times, servers with issues: jenkins-hbase4.apache.org,42073,1702191353745 at org.apache.hadoop.hbase.client.BatchErrors.makeException(BatchErrors.java:50) at org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.getErrors(AsyncRequestFutureImpl.java:1236) at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:434) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:389) at org.apache.hadoop.hbase.quotas.QuotaTableUtil.doGet(QuotaTableUtil.java:910) at org.apache.hadoop.hbase.quotas.QuotaUtil.fetchGlobalQuotas(QuotaUtil.java:374) at org.apache.hadoop.hbase.quotas.QuotaUtil.fetchNamespaceQuotas(QuotaUtil.java:341) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore$1.fetchEntries(QuotaCache.java:242) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetch(QuotaCache.java:333) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetchNamespaceQuotaState(QuotaCache.java:234) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.chore(QuotaCache.java:225) at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:107) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-12-10 07:01:55,110 WARN [regionserver/jenkins-hbase4:0.Chore.7] quotas.QuotaCache$QuotaRefresherChore(344): Unable to read namespace from quota table org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 2 actions: Timeout exceeded before call began. Meta requests may be slow, the operation timeout is too short for the number of requests, or the configured retries can't complete in the operation timeout.: 2 times, servers with issues: jenkins-hbase4.apache.org,37997,1702191353702 at org.apache.hadoop.hbase.client.BatchErrors.makeException(BatchErrors.java:50) at org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.getErrors(AsyncRequestFutureImpl.java:1236) at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:434) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:389) at org.apache.hadoop.hbase.quotas.QuotaTableUtil.doGet(QuotaTableUtil.java:910) at org.apache.hadoop.hbase.quotas.QuotaUtil.fetchGlobalQuotas(QuotaUtil.java:374) at org.apache.hadoop.hbase.quotas.QuotaUtil.fetchNamespaceQuotas(QuotaUtil.java:341) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore$1.fetchEntries(QuotaCache.java:242) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetch(QuotaCache.java:333) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetchNamespaceQuotaState(QuotaCache.java:234) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.chore(QuotaCache.java:225) at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:107) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-12-10 07:01:55,110 WARN [regionserver/jenkins-hbase4:0.Chore.2] quotas.QuotaCache$QuotaRefresherChore(344): Unable to read namespace from quota table org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 2 actions: Timeout exceeded before call began. Meta requests may be slow, the operation timeout is too short for the number of requests, or the configured retries can't complete in the operation timeout.: 2 times, servers with issues: jenkins-hbase4.apache.org,42073,1702191353745 at org.apache.hadoop.hbase.client.BatchErrors.makeException(BatchErrors.java:50) at org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.getErrors(AsyncRequestFutureImpl.java:1236) at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:434) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:389) at org.apache.hadoop.hbase.quotas.QuotaTableUtil.doGet(QuotaTableUtil.java:910) at org.apache.hadoop.hbase.quotas.QuotaUtil.fetchGlobalQuotas(QuotaUtil.java:374) at org.apache.hadoop.hbase.quotas.QuotaUtil.fetchNamespaceQuotas(QuotaUtil.java:341) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore$1.fetchEntries(QuotaCache.java:242) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetch(QuotaCache.java:333) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetchNamespaceQuotaState(QuotaCache.java:234) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.chore(QuotaCache.java:225) at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:107) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-12-10 07:01:55,110 WARN [regionserver/jenkins-hbase4:0.Chore.4] quotas.QuotaCache$QuotaRefresherChore(344): Unable to read namespace from quota table org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 2 actions: Timeout exceeded before call began. Meta requests may be slow, the operation timeout is too short for the number of requests, or the configured retries can't complete in the operation timeout.: 2 times, servers with issues: jenkins-hbase4.apache.org,42073,1702191353745 at org.apache.hadoop.hbase.client.BatchErrors.makeException(BatchErrors.java:50) at org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.getErrors(AsyncRequestFutureImpl.java:1236) at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:434) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:389) at org.apache.hadoop.hbase.quotas.QuotaTableUtil.doGet(QuotaTableUtil.java:910) at org.apache.hadoop.hbase.quotas.QuotaUtil.fetchGlobalQuotas(QuotaUtil.java:374) at org.apache.hadoop.hbase.quotas.QuotaUtil.fetchNamespaceQuotas(QuotaUtil.java:341) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore$1.fetchEntries(QuotaCache.java:242) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetch(QuotaCache.java:333) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetchNamespaceQuotaState(QuotaCache.java:234) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.chore(QuotaCache.java:225) at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:107) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-12-10 07:01:55,110 WARN [regionserver/jenkins-hbase4:0.Chore.1] quotas.QuotaCache$QuotaRefresherChore(344): Unable to read namespace from quota table org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 2 actions: Timeout exceeded before call began. Meta requests may be slow, the operation timeout is too short for the number of requests, or the configured retries can't complete in the operation timeout.: 2 times, servers with issues: jenkins-hbase4.apache.org,37997,1702191353702 at org.apache.hadoop.hbase.client.BatchErrors.makeException(BatchErrors.java:50) at org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.getErrors(AsyncRequestFutureImpl.java:1236) at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:434) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:389) at org.apache.hadoop.hbase.quotas.QuotaTableUtil.doGet(QuotaTableUtil.java:910) at org.apache.hadoop.hbase.quotas.QuotaUtil.fetchGlobalQuotas(QuotaUtil.java:374) at org.apache.hadoop.hbase.quotas.QuotaUtil.fetchNamespaceQuotas(QuotaUtil.java:341) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore$1.fetchEntries(QuotaCache.java:242) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetch(QuotaCache.java:333) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetchNamespaceQuotaState(QuotaCache.java:234) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.chore(QuotaCache.java:225) at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:107) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-12-10 07:01:55,110 WARN [regionserver/jenkins-hbase4:0.Chore.8] quotas.QuotaCache$QuotaRefresherChore(344): Unable to read namespace from quota table org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 2 actions: Timeout exceeded before call began. Meta requests may be slow, the operation timeout is too short for the number of requests, or the configured retries can't complete in the operation timeout.: 2 times, servers with issues: jenkins-hbase4.apache.org,42073,1702191353745 at org.apache.hadoop.hbase.client.BatchErrors.makeException(BatchErrors.java:50) at org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.getErrors(AsyncRequestFutureImpl.java:1236) at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:434) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:389) at org.apache.hadoop.hbase.quotas.QuotaTableUtil.doGet(QuotaTableUtil.java:910) at org.apache.hadoop.hbase.quotas.QuotaUtil.fetchGlobalQuotas(QuotaUtil.java:374) at org.apache.hadoop.hbase.quotas.QuotaUtil.fetchNamespaceQuotas(QuotaUtil.java:341) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore$1.fetchEntries(QuotaCache.java:242) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetch(QuotaCache.java:333) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetchNamespaceQuotaState(QuotaCache.java:234) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.chore(QuotaCache.java:225) at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:107) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-12-10 07:01:55,109 WARN [hconnection-0x214824e3-shared-pool-9] client.AsyncRequestFutureImpl(819): id=2, table=hbase:quota, attempt=1/46, failureCount=2ops, last exception=org.apache.hadoop.hbase.client.OperationTimeoutExceededException: Timeout exceeded before call began. Meta requests may be slow, the operation timeout is too short for the number of requests, or the configured retries can't complete in the operation timeout. on jenkins-hbase4.apache.org,37997,1702191353702, tracking started Sun Dec 10 07:01:54 UTC 2023; NOT retrying, failed=2 -- final attempt! 2023-12-10 07:01:55,109 WARN [hconnection-0x214824e3-shared-pool-11] client.AsyncRequestFutureImpl(819): id=2, table=hbase:quota, attempt=1/46, failureCount=2ops, last exception=org.apache.hadoop.hbase.client.OperationTimeoutExceededException: Timeout exceeded before call began. Meta requests may be slow, the operation timeout is too short for the number of requests, or the configured retries can't complete in the operation timeout. on jenkins-hbase4.apache.org,37997,1702191353702, tracking started Sun Dec 10 07:01:54 UTC 2023; NOT retrying, failed=2 -- final attempt! 2023-12-10 07:01:55,113 INFO [regionserver/jenkins-hbase4:0.Chore.5] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,125 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(184): QuotaCache 2023-12-10 07:01:55,126 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(185): {default=QuotaState(ts=0 bypass), TestNs=QuotaState(ts=1703831731973 bypass)} 2023-12-10 07:01:55,126 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(186): {TestQuotaAdmin0=QuotaState(ts=1705300337073 bypass), TestNs:TestTable=QuotaState(ts=1705300337073 bypass), TestQuotaAdmin2=QuotaState(ts=1705300337073 bypass)} 2023-12-10 07:01:55,127 INFO [regionserver/jenkins-hbase4:0.Chore.5] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:01:55,129 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(187): {jenkins=UserQuotaState(ts=1705300340173 [ default ])} 2023-12-10 07:01:55,131 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(188): {all=QuotaState(ts=1705300463673 bypass)} 2023-12-10 07:01:55,131 DEBUG [Listener at localhost/38369] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-12-10 07:01:55,134 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35846, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-12-10 07:01:55,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] ipc.CallRunner(144): callId: 122 service: ClientService methodName: Get size: 113 connection: 172.31.14.131:35846 deadline: 1705300473673, exception=org.apache.hadoop.hbase.exceptions.RegionMovedException: Region moved to: hostname=jenkins-hbase4.apache.org port=42073 startCode=1702191353745. As of locationSeqNum=2. 2023-12-10 07:01:55,387 DEBUG [Listener at localhost/38369] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-12-10 07:01:55,390 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44840, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-12-10 07:01:55,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 12sec, 0ms 2023-12-10 07:01:55,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 129 service: ClientService methodName: Get size: 114 connection: 172.31.14.131:44840 deadline: 1705300473673, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 12sec, 0ms 2023-12-10 07:01:55,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 12sec, 0ms 2023-12-10 07:01:55,646 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 131 service: ClientService methodName: Get size: 114 connection: 172.31.14.131:44840 deadline: 1705300473673, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 12sec, 0ms 2023-12-10 07:01:56,151 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 12sec, 0ms 2023-12-10 07:01:56,151 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 133 service: ClientService methodName: Get size: 114 connection: 172.31.14.131:44840 deadline: 1705300473673, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 12sec, 0ms 2023-12-10 07:01:56,265 DEBUG [master/jenkins-hbase4:0.Chore.1] zookeeper.ReadOnlyZKClient(139): Connect 0x4162393f to 127.0.0.1:51670 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-12-10 07:01:56,271 DEBUG [master/jenkins-hbase4:0.Chore.1] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1d0156f2, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-12-10 07:01:56,272 DEBUG [hconnection-0x6c2e9568-metaLookup-shared--pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-12-10 07:01:56,274 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35850, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-12-10 07:01:56,284 DEBUG [hconnection-0x6c2e9568-shared-pool-0] ipc.RpcConnection(122): Using SIMPLE authentication for service=ClientService, sasl=false 2023-12-10 07:01:56,285 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44844, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-12-10 07:01:56,910 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 12sec, 0ms 2023-12-10 07:01:56,910 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 135 service: ClientService methodName: Get size: 114 connection: 172.31.14.131:44840 deadline: 1705300473673, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 12sec, 0ms 2023-12-10 07:01:58,164 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 12sec, 0ms 2023-12-10 07:01:58,164 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 137 service: ClientService methodName: Get size: 114 connection: 172.31.14.131:44840 deadline: 1705300473673, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 12sec, 0ms 2023-12-10 07:02:00,672 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 12sec, 0ms 2023-12-10 07:02:00,672 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 139 service: ClientService methodName: Get size: 114 connection: 172.31.14.131:44840 deadline: 1705300473673, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 12sec, 0ms 2023-12-10 07:02:05,718 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 12sec, 0ms 2023-12-10 07:02:05,718 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 141 service: ClientService methodName: Get size: 114 connection: 172.31.14.131:44840 deadline: 1705300473673, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 12sec, 0ms 2023-12-10 07:02:05,720 DEBUG [Listener at localhost/38369] client.RpcRetryingCallerImpl(129): Call exception, tries=6, retries=16, started=0 ms ago, cancelled=false, msg=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 12sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) , details=row 'row-5' on table 'TestQuotaAdmin0' at region=TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37., hostname=jenkins-hbase4.apache.org,42073,1702191353745, seqNum=2, see https://s.apache.org/timeout, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 12sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.GeneratedConstructorAccessor45.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:276) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:261) at org.apache.hadoop.hbase.client.RegionServerCallable.call(RegionServerCallable.java:126) at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:104) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:370) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:343) at org.apache.hadoop.hbase.quotas.ThrottleQuotaTestUtil.doGets(ThrottleQuotaTestUtil.java:95) at org.apache.hadoop.hbase.quotas.TestClusterScopeQuotaThrottle.testUserNamespaceClusterScopeQuota(TestClusterScopeQuotaThrottle.java:199) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.quotas.RpcThrottlingException): org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 12sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:381) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:87) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:415) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:411) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:193) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:214) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-12-10 07:02:05,720 ERROR [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(100): get failed after nRetries=5 java.net.SocketTimeoutException: callTimeout=10000, callDuration=10001: org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 12sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) row 'row-5' on table 'TestQuotaAdmin0' at region=TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37., hostname=jenkins-hbase4.apache.org,42073,1702191353745, seqNum=2 at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:156) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:370) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:343) at org.apache.hadoop.hbase.quotas.ThrottleQuotaTestUtil.doGets(ThrottleQuotaTestUtil.java:95) at org.apache.hadoop.hbase.quotas.TestClusterScopeQuotaThrottle.testUserNamespaceClusterScopeQuota(TestClusterScopeQuotaThrottle.java:199) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.quotas.RpcThrottlingException: org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 12sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.GeneratedConstructorAccessor45.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:276) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:261) at org.apache.hadoop.hbase.client.RegionServerCallable.call(RegionServerCallable.java:126) at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:104) ... 29 more Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.quotas.RpcThrottlingException): org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 12sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:381) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:87) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:415) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:411) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:193) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:214) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-12-10 07:02:05,728 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=1 numReads=0 numScans=0: number of write requests exceeded - wait 10sec, 0ms 2023-12-10 07:02:05,728 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 148 service: ClientService methodName: Mutate size: 142 connection: 172.31.14.131:44840 deadline: 1705300473673, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 10sec, 0ms 2023-12-10 07:02:05,981 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=1 numReads=0 numScans=0: number of write requests exceeded - wait 10sec, 0ms 2023-12-10 07:02:05,981 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 150 service: ClientService methodName: Mutate size: 142 connection: 172.31.14.131:44840 deadline: 1705300473673, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 10sec, 0ms 2023-12-10 07:02:06,487 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=1 numReads=0 numScans=0: number of write requests exceeded - wait 10sec, 0ms 2023-12-10 07:02:06,488 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 152 service: ClientService methodName: Mutate size: 142 connection: 172.31.14.131:44840 deadline: 1705300473673, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 10sec, 0ms 2023-12-10 07:02:07,242 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=1 numReads=0 numScans=0: number of write requests exceeded - wait 10sec, 0ms 2023-12-10 07:02:07,243 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 154 service: ClientService methodName: Mutate size: 142 connection: 172.31.14.131:44840 deadline: 1705300473673, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 10sec, 0ms 2023-12-10 07:02:08,496 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=1 numReads=0 numScans=0: number of write requests exceeded - wait 10sec, 0ms 2023-12-10 07:02:08,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 156 service: ClientService methodName: Mutate size: 142 connection: 172.31.14.131:44840 deadline: 1705300473673, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 10sec, 0ms 2023-12-10 07:02:11,003 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=1 numReads=0 numScans=0: number of write requests exceeded - wait 10sec, 0ms 2023-12-10 07:02:11,003 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 158 service: ClientService methodName: Mutate size: 142 connection: 172.31.14.131:44840 deadline: 1705300473673, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 10sec, 0ms 2023-12-10 07:02:16,051 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=1 numReads=0 numScans=0: number of write requests exceeded - wait 10sec, 0ms 2023-12-10 07:02:16,051 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 160 service: ClientService methodName: Mutate size: 142 connection: 172.31.14.131:44840 deadline: 1705300473673, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 10sec, 0ms 2023-12-10 07:02:16,052 DEBUG [Listener at localhost/38369] client.RpcRetryingCallerImpl(129): Call exception, tries=6, retries=16, started=0 ms ago, cancelled=false, msg=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 10sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumWriteRequestsExceeded(RpcThrottlingException.java:104) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:158) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:170) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2942) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) , details=row 'row-6' on table 'TestQuotaAdmin0' at region=TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37., hostname=jenkins-hbase4.apache.org,42073,1702191353745, seqNum=2, see https://s.apache.org/timeout, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 10sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumWriteRequestsExceeded(RpcThrottlingException.java:104) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:158) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:170) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2942) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.GeneratedConstructorAccessor45.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:276) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:261) at org.apache.hadoop.hbase.client.RegionServerCallable.call(RegionServerCallable.java:126) at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:104) at org.apache.hadoop.hbase.client.HTable.put(HTable.java:512) at org.apache.hadoop.hbase.quotas.ThrottleQuotaTestUtil.doPuts(ThrottleQuotaTestUtil.java:71) at org.apache.hadoop.hbase.quotas.ThrottleQuotaTestUtil.doPuts(ThrottleQuotaTestUtil.java:54) at org.apache.hadoop.hbase.quotas.TestClusterScopeQuotaThrottle.testUserNamespaceClusterScopeQuota(TestClusterScopeQuotaThrottle.java:200) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.quotas.RpcThrottlingException): org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 10sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumWriteRequestsExceeded(RpcThrottlingException.java:104) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:158) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:170) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2942) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:381) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:87) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:415) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:411) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:193) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:214) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-12-10 07:02:16,052 ERROR [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(76): put failed after nRetries=6 java.net.SocketTimeoutException: callTimeout=10000, callDuration=10023: org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 10sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumWriteRequestsExceeded(RpcThrottlingException.java:104) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:158) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:170) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2942) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) row 'row-6' on table 'TestQuotaAdmin0' at region=TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37., hostname=jenkins-hbase4.apache.org,42073,1702191353745, seqNum=2 at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:156) at org.apache.hadoop.hbase.client.HTable.put(HTable.java:512) at org.apache.hadoop.hbase.quotas.ThrottleQuotaTestUtil.doPuts(ThrottleQuotaTestUtil.java:71) at org.apache.hadoop.hbase.quotas.ThrottleQuotaTestUtil.doPuts(ThrottleQuotaTestUtil.java:54) at org.apache.hadoop.hbase.quotas.TestClusterScopeQuotaThrottle.testUserNamespaceClusterScopeQuota(TestClusterScopeQuotaThrottle.java:200) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.quotas.RpcThrottlingException: org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 10sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumWriteRequestsExceeded(RpcThrottlingException.java:104) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:158) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:170) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2942) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.GeneratedConstructorAccessor45.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:276) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:261) at org.apache.hadoop.hbase.client.RegionServerCallable.call(RegionServerCallable.java:126) at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:104) ... 29 more Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.quotas.RpcThrottlingException): org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 10sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumWriteRequestsExceeded(RpcThrottlingException.java:104) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:158) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:170) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2942) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:381) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:87) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:415) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:411) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:193) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:214) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-12-10 07:02:16,053 DEBUG [Listener at localhost/38369] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-12-10 07:02:16,056 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47334, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-12-10 07:02:16,059 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] ipc.RpcConnection(122): Using SIMPLE authentication for service=MasterService, sasl=false 2023-12-10 07:02:16,060 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47346, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=MasterService 2023-12-10 07:02:16,063 INFO [regionserver/jenkins-hbase4:0.Chore.5] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:02:16,309 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(184): QuotaCache 2023-12-10 07:02:16,310 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(185): {default=QuotaState(ts=1705304063673 bypass), TestNs=QuotaState(ts=1705304063673 bypass)} 2023-12-10 07:02:16,310 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(186): {TestQuotaAdmin0=QuotaState(ts=0 bypass), TestNs:TestTable=QuotaState(ts=1705304063673 bypass), TestQuotaAdmin2=QuotaState(ts=1705304063673 bypass), TestQuotaAdmin1=QuotaState(ts=1705304063673 bypass)} 2023-12-10 07:02:16,310 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(187): {jenkins=UserQuotaState(ts=0 bypass)} 2023-12-10 07:02:16,310 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(188): {all=QuotaState(ts=1705304063673 bypass)} 2023-12-10 07:02:16,560 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(184): QuotaCache 2023-12-10 07:02:16,560 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(185): {default=QuotaState(ts=1705304063673 bypass), TestNs=QuotaState(ts=1705304063673 bypass)} 2023-12-10 07:02:16,560 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(186): {TestQuotaAdmin0=QuotaState(ts=1705304063673 bypass), TestNs:TestTable=QuotaState(ts=1705304063673 bypass), TestQuotaAdmin2=QuotaState(ts=1705304063673 bypass)} 2023-12-10 07:02:16,560 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(187): {jenkins=UserQuotaState(ts=1705304063673 bypass)} 2023-12-10 07:02:16,560 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(188): {all=QuotaState(ts=1705304063673 bypass)} 2023-12-10 07:02:16,573 INFO [Listener at localhost/38369] hbase.ResourceChecker(175): after: quotas.TestClusterScopeQuotaThrottle#testUserNamespaceClusterScopeQuota Thread=287 (was 298), OpenFileDescriptor=598 (was 627), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=189 (was 206), ProcessCount=169 (was 171), AvailableMemoryMB=5052 (was 5677) 2023-12-10 07:02:16,585 INFO [Listener at localhost/38369] hbase.ResourceChecker(147): before: quotas.TestClusterScopeQuotaThrottle#testUserClusterScopeQuota Thread=288, OpenFileDescriptor=598, MaxFileDescriptor=60000, SystemLoadAverage=174, ProcessCount=169, AvailableMemoryMB=5052 2023-12-10 07:02:16,596 INFO [regionserver/jenkins-hbase4:0.Chore.5] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:02:16,761 INFO [regionserver/jenkins-hbase4:0.Chore.7] regionserver.HRegionServer$PeriodicMemStoreFlusher(1921): MemstoreFlusherChore requesting flush of hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b. because dca2e6ecbeef49fb8929fd36dd74df8b/q has an old edit so flush to free WALs after random delay 138947 ms 2023-12-10 07:02:16,761 INFO [regionserver/jenkins-hbase4:0.Chore.7] regionserver.HRegionServer$PeriodicMemStoreFlusher(1921): MemstoreFlusherChore requesting flush of TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37. because 738cc1ffb58c4f3e8fbde456163cdb37/cf has an old edit so flush to free WALs after random delay 225337 ms 2023-12-10 07:02:16,848 INFO [regionserver/jenkins-hbase4:0.Chore.5] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:02:16,849 WARN [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=42073] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1705307755473","responsesize":"63","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":51300,"client":"172.31.14.131:41370","queuetimems":500,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 07:02:16,849 WARN [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=42073] ipc.RpcServer(528): (responseTooSlow): {"call":"Get(org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$GetRequest)","starttimems":"1705307757673","responsesize":"63","method":"Get","param":"region= hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b., row=u.jenkins","processingtimems":48800,"client":"172.31.14.131:41370","queuetimems":600,"class":"MiniHBaseClusterRegionServer"} 2023-12-10 07:02:16,850 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(184): QuotaCache 2023-12-10 07:02:16,850 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(185): {default=QuotaState(ts=1705307686673 bypass), TestNs=QuotaState(ts=1705307686673 bypass)} 2023-12-10 07:02:16,850 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(186): {TestQuotaAdmin0=QuotaState(ts=1705307724273 bypass), TestNs:TestTable=QuotaState(ts=1705307724273 bypass), TestQuotaAdmin2=QuotaState(ts=1705307724273 bypass), TestQuotaAdmin1=QuotaState(ts=1705307724273 bypass)} 2023-12-10 07:02:16,851 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(187): {jenkins=UserQuotaState(ts=1705307752373 global-limiter)} 2023-12-10 07:02:16,851 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(188): {all=QuotaState(ts=1705307663673 bypass)} 2023-12-10 07:02:16,858 INFO [regionserver/jenkins-hbase4:0.Chore.5] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:02:16,916 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2489): Flush column family: q of dca2e6ecbeef49fb8929fd36dd74df8b because time of oldest edit=1705300463673 is > 3600000 from now =1705307834273 2023-12-10 07:02:16,916 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing dca2e6ecbeef49fb8929fd36dd74df8b 1/2 column families, dataSize=138 B heapSize=872 B; q={dataSize=138 B, heapSize=616 B, offHeapSize=0 B} 2023-12-10 07:02:16,933 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=138 B at sequenceid=23 (bloomFilter=true), to=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/.tmp/q/85bbf866636945bb818d554db03964ab 2023-12-10 07:02:16,942 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/.tmp/q/85bbf866636945bb818d554db03964ab as hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/q/85bbf866636945bb818d554db03964ab 2023-12-10 07:02:16,948 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/q/85bbf866636945bb818d554db03964ab, entries=2, sequenceid=23, filesize=4.8 K 2023-12-10 07:02:16,950 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~138 B/138, heapSize ~600 B/600, currentSize=0 B/0 for dca2e6ecbeef49fb8929fd36dd74df8b in 0ms, sequenceid=23, compaction requested=false 2023-12-10 07:02:16,950 DEBUG [MemStoreFlusher.0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:quota' 2023-12-10 07:02:16,950 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for dca2e6ecbeef49fb8929fd36dd74df8b: 2023-12-10 07:02:17,112 INFO [regionserver/jenkins-hbase4:0.Chore.5] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:02:17,121 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(184): QuotaCache 2023-12-10 07:02:17,121 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(185): {default=QuotaState(ts=1705307865273 bypass), TestNs=QuotaState(ts=1705307865273 bypass)} 2023-12-10 07:02:17,121 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(186): {TestQuotaAdmin0=QuotaState(ts=1705307870573 bypass), TestNs:TestTable=QuotaState(ts=1705307870573 bypass), TestQuotaAdmin2=QuotaState(ts=1705307870573 bypass), TestQuotaAdmin1=QuotaState(ts=1705307870573 bypass)} 2023-12-10 07:02:17,121 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(187): {jenkins=UserQuotaState(ts=1705307864573 global-limiter)} 2023-12-10 07:02:17,121 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(188): {all=QuotaState(ts=1705307955573 bypass)} 2023-12-10 07:02:17,123 INFO [regionserver/jenkins-hbase4:0.Chore.5] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:02:17,129 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=1 numReads=0 numScans=0: number of write requests exceeded - wait 10sec, 0ms 2023-12-10 07:02:17,129 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 173 service: ClientService methodName: Mutate size: 142 connection: 172.31.14.131:44840 deadline: 1705307965573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 10sec, 0ms 2023-12-10 07:02:17,152 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing 738cc1ffb58c4f3e8fbde456163cdb37 1/1 column families, dataSize=408 B heapSize=1.56 KB 2023-12-10 07:02:17,170 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=408 B at sequenceid=19 (bloomFilter=false), to=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin0/738cc1ffb58c4f3e8fbde456163cdb37/.tmp/cf/245239aeae9b492dba637fad4f9883c5 2023-12-10 07:02:17,177 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin0/738cc1ffb58c4f3e8fbde456163cdb37/.tmp/cf/245239aeae9b492dba637fad4f9883c5 as hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin0/738cc1ffb58c4f3e8fbde456163cdb37/cf/245239aeae9b492dba637fad4f9883c5 2023-12-10 07:02:17,183 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin0/738cc1ffb58c4f3e8fbde456163cdb37/cf/245239aeae9b492dba637fad4f9883c5, entries=6, sequenceid=19, filesize=4.7 K 2023-12-10 07:02:17,184 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~408 B/408, heapSize ~1.55 KB/1584, currentSize=0 B/0 for 738cc1ffb58c4f3e8fbde456163cdb37 in 0ms, sequenceid=19, compaction requested=false 2023-12-10 07:02:17,184 DEBUG [MemStoreFlusher.0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'TestQuotaAdmin0' 2023-12-10 07:02:17,185 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for 738cc1ffb58c4f3e8fbde456163cdb37: 2023-12-10 07:02:17,381 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=1 numReads=0 numScans=0: number of write requests exceeded - wait 10sec, 0ms 2023-12-10 07:02:17,381 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 175 service: ClientService methodName: Mutate size: 142 connection: 172.31.14.131:44840 deadline: 1705307965573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 10sec, 0ms 2023-12-10 07:02:17,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=1 numReads=0 numScans=0: number of write requests exceeded - wait 10sec, 0ms 2023-12-10 07:02:17,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 177 service: ClientService methodName: Mutate size: 142 connection: 172.31.14.131:44840 deadline: 1705307965573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 10sec, 0ms 2023-12-10 07:02:18,644 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=1 numReads=0 numScans=0: number of write requests exceeded - wait 10sec, 0ms 2023-12-10 07:02:18,644 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 179 service: ClientService methodName: Mutate size: 142 connection: 172.31.14.131:44840 deadline: 1705307965573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 10sec, 0ms 2023-12-10 07:02:19,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=1 numReads=0 numScans=0: number of write requests exceeded - wait 10sec, 0ms 2023-12-10 07:02:19,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 181 service: ClientService methodName: Mutate size: 142 connection: 172.31.14.131:44840 deadline: 1705307965573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 10sec, 0ms 2023-12-10 07:02:22,429 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=1 numReads=0 numScans=0: number of write requests exceeded - wait 10sec, 0ms 2023-12-10 07:02:22,429 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 183 service: ClientService methodName: Mutate size: 142 connection: 172.31.14.131:44840 deadline: 1705307965573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 10sec, 0ms 2023-12-10 07:02:27,432 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=1 numReads=0 numScans=0: number of write requests exceeded - wait 10sec, 0ms 2023-12-10 07:02:27,432 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 185 service: ClientService methodName: Mutate size: 142 connection: 172.31.14.131:44840 deadline: 1705307965573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 10sec, 0ms 2023-12-10 07:02:27,434 DEBUG [Listener at localhost/38369] client.RpcRetryingCallerImpl(129): Call exception, tries=6, retries=16, started=0 ms ago, cancelled=false, msg=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 10sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumWriteRequestsExceeded(RpcThrottlingException.java:104) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:158) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:170) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2942) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) , details=row 'row-6' on table 'TestQuotaAdmin0' at region=TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37., hostname=jenkins-hbase4.apache.org,42073,1702191353745, seqNum=2, see https://s.apache.org/timeout, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 10sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumWriteRequestsExceeded(RpcThrottlingException.java:104) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:158) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:170) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2942) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.GeneratedConstructorAccessor45.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:276) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:261) at org.apache.hadoop.hbase.client.RegionServerCallable.call(RegionServerCallable.java:126) at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:104) at org.apache.hadoop.hbase.client.HTable.put(HTable.java:512) at org.apache.hadoop.hbase.quotas.ThrottleQuotaTestUtil.doPuts(ThrottleQuotaTestUtil.java:71) at org.apache.hadoop.hbase.quotas.ThrottleQuotaTestUtil.doPuts(ThrottleQuotaTestUtil.java:54) at org.apache.hadoop.hbase.quotas.TestClusterScopeQuotaThrottle.testUserClusterScopeQuota(TestClusterScopeQuotaThrottle.java:178) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.quotas.RpcThrottlingException): org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 10sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumWriteRequestsExceeded(RpcThrottlingException.java:104) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:158) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:170) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2942) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:381) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:87) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:415) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:411) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:193) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:214) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-12-10 07:02:27,434 ERROR [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(76): put failed after nRetries=6 java.net.SocketTimeoutException: callTimeout=10000, callDuration=10041: org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 10sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumWriteRequestsExceeded(RpcThrottlingException.java:104) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:158) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:170) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2942) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) row 'row-6' on table 'TestQuotaAdmin0' at region=TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37., hostname=jenkins-hbase4.apache.org,42073,1702191353745, seqNum=2 at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:156) at org.apache.hadoop.hbase.client.HTable.put(HTable.java:512) at org.apache.hadoop.hbase.quotas.ThrottleQuotaTestUtil.doPuts(ThrottleQuotaTestUtil.java:71) at org.apache.hadoop.hbase.quotas.ThrottleQuotaTestUtil.doPuts(ThrottleQuotaTestUtil.java:54) at org.apache.hadoop.hbase.quotas.TestClusterScopeQuotaThrottle.testUserClusterScopeQuota(TestClusterScopeQuotaThrottle.java:178) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.quotas.RpcThrottlingException: org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 10sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumWriteRequestsExceeded(RpcThrottlingException.java:104) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:158) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:170) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2942) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.GeneratedConstructorAccessor45.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:276) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:261) at org.apache.hadoop.hbase.client.RegionServerCallable.call(RegionServerCallable.java:126) at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:104) ... 29 more Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.quotas.RpcThrottlingException): org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 10sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumWriteRequestsExceeded(RpcThrottlingException.java:104) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:158) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:170) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2942) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:381) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:87) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:415) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:411) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:193) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:214) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-12-10 07:02:27,440 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 20sec, 0ms 2023-12-10 07:02:27,440 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 189 service: ClientService methodName: Get size: 114 connection: 172.31.14.131:44840 deadline: 1705307965573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 20sec, 0ms 2023-12-10 07:02:27,692 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 20sec, 0ms 2023-12-10 07:02:27,692 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 191 service: ClientService methodName: Get size: 114 connection: 172.31.14.131:44840 deadline: 1705307965573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 20sec, 0ms 2023-12-10 07:02:28,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 20sec, 0ms 2023-12-10 07:02:28,194 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 193 service: ClientService methodName: Get size: 114 connection: 172.31.14.131:44840 deadline: 1705307965573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 20sec, 0ms 2023-12-10 07:02:28,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 20sec, 0ms 2023-12-10 07:02:28,950 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 195 service: ClientService methodName: Get size: 114 connection: 172.31.14.131:44840 deadline: 1705307965573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 20sec, 0ms 2023-12-10 07:02:30,203 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 20sec, 0ms 2023-12-10 07:02:30,203 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 197 service: ClientService methodName: Get size: 114 connection: 172.31.14.131:44840 deadline: 1705307965573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 20sec, 0ms 2023-12-10 07:02:32,716 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 20sec, 0ms 2023-12-10 07:02:32,716 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 199 service: ClientService methodName: Get size: 114 connection: 172.31.14.131:44840 deadline: 1705307965573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 20sec, 0ms 2023-12-10 07:02:37,730 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 20sec, 0ms 2023-12-10 07:02:37,730 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 201 service: ClientService methodName: Get size: 114 connection: 172.31.14.131:44840 deadline: 1705307965573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 20sec, 0ms 2023-12-10 07:02:37,732 DEBUG [Listener at localhost/38369] client.RpcRetryingCallerImpl(129): Call exception, tries=6, retries=16, started=0 ms ago, cancelled=false, msg=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 20sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) , details=row 'row-3' on table 'TestQuotaAdmin0' at region=TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37., hostname=jenkins-hbase4.apache.org,42073,1702191353745, seqNum=2, see https://s.apache.org/timeout, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 20sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.GeneratedConstructorAccessor45.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:276) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:261) at org.apache.hadoop.hbase.client.RegionServerCallable.call(RegionServerCallable.java:126) at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:104) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:370) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:343) at org.apache.hadoop.hbase.quotas.ThrottleQuotaTestUtil.doGets(ThrottleQuotaTestUtil.java:95) at org.apache.hadoop.hbase.quotas.TestClusterScopeQuotaThrottle.testUserClusterScopeQuota(TestClusterScopeQuotaThrottle.java:179) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.quotas.RpcThrottlingException): org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 20sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:381) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:87) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:415) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:411) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:193) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:214) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-12-10 07:02:37,732 ERROR [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(100): get failed after nRetries=3 java.net.SocketTimeoutException: callTimeout=10000, callDuration=10078: org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 20sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) row 'row-3' on table 'TestQuotaAdmin0' at region=TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37., hostname=jenkins-hbase4.apache.org,42073,1702191353745, seqNum=2 at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:156) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:370) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:343) at org.apache.hadoop.hbase.quotas.ThrottleQuotaTestUtil.doGets(ThrottleQuotaTestUtil.java:95) at org.apache.hadoop.hbase.quotas.TestClusterScopeQuotaThrottle.testUserClusterScopeQuota(TestClusterScopeQuotaThrottle.java:179) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.quotas.RpcThrottlingException: org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 20sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.GeneratedConstructorAccessor45.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:276) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:261) at org.apache.hadoop.hbase.client.RegionServerCallable.call(RegionServerCallable.java:126) at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:104) ... 29 more Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.quotas.RpcThrottlingException): org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 20sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:381) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:87) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:415) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:411) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:193) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:214) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-12-10 07:02:37,745 INFO [regionserver/jenkins-hbase4:0.Chore.5] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:02:37,772 INFO [regionserver/jenkins-hbase4:0.Chore.4] regionserver.HRegionServer$PeriodicMemStoreFlusher(1921): MemstoreFlusherChore requesting flush of hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b. because dca2e6ecbeef49fb8929fd36dd74df8b/q has an old edit so flush to free WALs after random delay 166397 ms 2023-12-10 07:02:37,985 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(184): QuotaCache 2023-12-10 07:02:37,985 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(185): {default=QuotaState(ts=1705311555573 bypass), TestNs=QuotaState(ts=1705311555573 bypass)} 2023-12-10 07:02:37,985 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(186): {TestQuotaAdmin0=QuotaState(ts=1705311555573 bypass), TestNs:TestTable=QuotaState(ts=1705311555573 bypass), TestQuotaAdmin2=QuotaState(ts=1705311555573 bypass), TestQuotaAdmin1=QuotaState(ts=1705311555573 bypass)} 2023-12-10 07:02:37,985 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(187): {jenkins=UserQuotaState(ts=1705311555573 bypass)} 2023-12-10 07:02:37,985 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(188): {all=QuotaState(ts=1705311555573 bypass)} 2023-12-10 07:02:38,236 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(184): QuotaCache 2023-12-10 07:02:38,236 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(185): {default=QuotaState(ts=1705311555573 bypass), TestNs=QuotaState(ts=1705311555573 bypass)} 2023-12-10 07:02:38,236 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(186): {TestQuotaAdmin0=QuotaState(ts=1705311555573 bypass), TestNs:TestTable=QuotaState(ts=1705311555573 bypass), TestQuotaAdmin2=QuotaState(ts=1705311555573 bypass), TestQuotaAdmin1=QuotaState(ts=1705311555573 bypass)} 2023-12-10 07:02:38,236 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(187): {jenkins=UserQuotaState(ts=1705311555573 bypass)} 2023-12-10 07:02:38,236 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(188): {all=QuotaState(ts=1705311555573 bypass)} 2023-12-10 07:02:38,248 INFO [Listener at localhost/38369] hbase.ResourceChecker(175): after: quotas.TestClusterScopeQuotaThrottle#testUserClusterScopeQuota Thread=289 (was 288) - Thread LEAK? -, OpenFileDescriptor=612 (was 598) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=125 (was 174), ProcessCount=168 (was 169), AvailableMemoryMB=5051 (was 5052) 2023-12-10 07:02:38,258 INFO [Listener at localhost/38369] hbase.ResourceChecker(147): before: quotas.TestClusterScopeQuotaThrottle#testTableClusterScopeQuota Thread=289, OpenFileDescriptor=612, MaxFileDescriptor=60000, SystemLoadAverage=125, ProcessCount=168, AvailableMemoryMB=5051 2023-12-10 07:02:38,270 INFO [regionserver/jenkins-hbase4:0.Chore.5] client.AsyncRequestFutureImpl(1207): #2, waiting for 2 actions to finish on table: hbase:quota 2023-12-10 07:02:38,300 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2489): Flush column family: q of dca2e6ecbeef49fb8929fd36dd74df8b because time of oldest edit=1705307955573 is > 3600000 from now =1705315155573 2023-12-10 07:02:38,300 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2489): Flush column family: u of dca2e6ecbeef49fb8929fd36dd74df8b because time of oldest edit=1705307955573 is > 3600000 from now =1705315155573 2023-12-10 07:02:38,300 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing dca2e6ecbeef49fb8929fd36dd74df8b 2/2 column families, dataSize=114 B heapSize=848 B 2023-12-10 07:02:38,316 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=84 B at sequenceid=28 (bloomFilter=true), to=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/.tmp/q/5a235b0bca254939bebe65b345c9c824 2023-12-10 07:02:38,323 INFO [MemStoreFlusher.0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5a235b0bca254939bebe65b345c9c824 2023-12-10 07:02:38,336 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=30 B at sequenceid=28 (bloomFilter=true), to=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/.tmp/u/b19cbfab0afa44cf9d69587dd125d7ab 2023-12-10 07:02:38,341 INFO [MemStoreFlusher.0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b19cbfab0afa44cf9d69587dd125d7ab 2023-12-10 07:02:38,342 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/.tmp/q/5a235b0bca254939bebe65b345c9c824 as hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/q/5a235b0bca254939bebe65b345c9c824 2023-12-10 07:02:38,347 INFO [MemStoreFlusher.0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5a235b0bca254939bebe65b345c9c824 2023-12-10 07:02:38,348 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/q/5a235b0bca254939bebe65b345c9c824, entries=2, sequenceid=28, filesize=5.0 K 2023-12-10 07:02:38,348 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/.tmp/u/b19cbfab0afa44cf9d69587dd125d7ab as hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/u/b19cbfab0afa44cf9d69587dd125d7ab 2023-12-10 07:02:38,354 INFO [MemStoreFlusher.0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b19cbfab0afa44cf9d69587dd125d7ab 2023-12-10 07:02:38,354 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/u/b19cbfab0afa44cf9d69587dd125d7ab, entries=1, sequenceid=28, filesize=4.9 K 2023-12-10 07:02:38,355 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~114 B/114, heapSize ~816 B/816, currentSize=0 B/0 for dca2e6ecbeef49fb8929fd36dd74df8b in 0ms, sequenceid=28, compaction requested=false 2023-12-10 07:02:38,355 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for dca2e6ecbeef49fb8929fd36dd74df8b: 2023-12-10 07:02:38,514 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(184): QuotaCache 2023-12-10 07:02:38,514 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(185): {default=QuotaState(ts=1705315155573 bypass), TestNs=QuotaState(ts=1705315155573 bypass)} 2023-12-10 07:02:38,515 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(186): {TestNs:TestTable=QuotaState(ts=1705315155573 TimeBasedLimiter( readReqs=AverageIntervalRateLimiter(avail=10 limit=10 tunit=3600000))), TestQuotaAdmin1=QuotaState(ts=1705315155573 bypass)} 2023-12-10 07:02:38,515 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(187): {} 2023-12-10 07:02:38,516 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(188): {all=QuotaState(ts=1705315155573 bypass)} 2023-12-10 07:02:38,766 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(184): QuotaCache 2023-12-10 07:02:38,767 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(185): {default=QuotaState(ts=1705315155573 bypass), TestNs=QuotaState(ts=1705315155573 bypass)} 2023-12-10 07:02:38,768 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(186): {TestQuotaAdmin0=QuotaState(ts=1705315155573 bypass), TestNs:TestTable=QuotaState(ts=1705315155573 TimeBasedLimiter( readReqs=AverageIntervalRateLimiter(avail=10 limit=10 tunit=3600000))), TestQuotaAdmin2=QuotaState(ts=1705315155573 bypass)} 2023-12-10 07:02:38,768 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(187): {} 2023-12-10 07:02:38,768 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(188): {all=QuotaState(ts=1705315155573 bypass)} 2023-12-10 07:02:38,773 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestNs:TestTable numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 07:02:38,773 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] ipc.CallRunner(144): callId: 216 service: ClientService methodName: Get size: 117 connection: 172.31.14.131:35846 deadline: 1705315165573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 07:02:39,026 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestNs:TestTable numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 07:02:39,026 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] ipc.CallRunner(144): callId: 218 service: ClientService methodName: Get size: 117 connection: 172.31.14.131:35846 deadline: 1705315165573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 07:02:39,530 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestNs:TestTable numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 07:02:39,530 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] ipc.CallRunner(144): callId: 220 service: ClientService methodName: Get size: 117 connection: 172.31.14.131:35846 deadline: 1705315165573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 07:02:40,286 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestNs:TestTable numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 07:02:40,287 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] ipc.CallRunner(144): callId: 222 service: ClientService methodName: Get size: 117 connection: 172.31.14.131:35846 deadline: 1705315165573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 07:02:41,540 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestNs:TestTable numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 07:02:41,540 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] ipc.CallRunner(144): callId: 224 service: ClientService methodName: Get size: 117 connection: 172.31.14.131:35846 deadline: 1705315165573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 07:02:44,066 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestNs:TestTable numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 07:02:44,066 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] ipc.CallRunner(144): callId: 226 service: ClientService methodName: Get size: 117 connection: 172.31.14.131:35846 deadline: 1705315165573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 07:02:49,109 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestNs:TestTable numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 07:02:49,110 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] ipc.CallRunner(144): callId: 228 service: ClientService methodName: Get size: 117 connection: 172.31.14.131:35846 deadline: 1705315165573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 07:02:49,111 DEBUG [Listener at localhost/38369] client.RpcRetryingCallerImpl(129): Call exception, tries=6, retries=16, started=0 ms ago, cancelled=false, msg=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) , details=row 'row-10' on table 'TestNs:TestTable' at region=TestNs:TestTable,1,1702191359393.79758411fb41135ec4572c165c4e2dc5., hostname=jenkins-hbase4.apache.org,37997,1702191353702, seqNum=2, see https://s.apache.org/timeout, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.GeneratedConstructorAccessor45.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:276) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:261) at org.apache.hadoop.hbase.client.RegionServerCallable.call(RegionServerCallable.java:126) at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:104) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:370) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:343) at org.apache.hadoop.hbase.quotas.ThrottleQuotaTestUtil.doGets(ThrottleQuotaTestUtil.java:95) at org.apache.hadoop.hbase.quotas.TestClusterScopeQuotaThrottle.testTableClusterScopeQuota(TestClusterScopeQuotaThrottle.java:151) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.quotas.RpcThrottlingException): org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:381) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:87) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:415) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:411) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:193) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:214) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-12-10 07:02:49,111 ERROR [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(100): get failed after nRetries=10 java.net.SocketTimeoutException: callTimeout=10000, callDuration=10077: org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) row 'row-10' on table 'TestNs:TestTable' at region=TestNs:TestTable,1,1702191359393.79758411fb41135ec4572c165c4e2dc5., hostname=jenkins-hbase4.apache.org,37997,1702191353702, seqNum=2 at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:156) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:370) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:343) at org.apache.hadoop.hbase.quotas.ThrottleQuotaTestUtil.doGets(ThrottleQuotaTestUtil.java:95) at org.apache.hadoop.hbase.quotas.TestClusterScopeQuotaThrottle.testTableClusterScopeQuota(TestClusterScopeQuotaThrottle.java:151) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.quotas.RpcThrottlingException: org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.GeneratedConstructorAccessor45.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:276) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:261) at org.apache.hadoop.hbase.client.RegionServerCallable.call(RegionServerCallable.java:126) at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:104) ... 29 more Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.quotas.RpcThrottlingException): org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:381) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:87) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:415) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:411) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:193) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:214) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-12-10 07:02:49,112 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestNs:TestTable numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 07:02:49,112 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] ipc.CallRunner(144): callId: 229 service: ClientService methodName: Get size: 116 connection: 172.31.14.131:35846 deadline: 1705315165573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 07:02:49,366 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestNs:TestTable numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 07:02:49,366 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] ipc.CallRunner(144): callId: 231 service: ClientService methodName: Get size: 116 connection: 172.31.14.131:35846 deadline: 1705315165573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 07:02:49,871 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestNs:TestTable numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 07:02:49,871 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] ipc.CallRunner(144): callId: 233 service: ClientService methodName: Get size: 116 connection: 172.31.14.131:35846 deadline: 1705315165573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 07:02:50,629 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestNs:TestTable numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 07:02:50,629 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] ipc.CallRunner(144): callId: 235 service: ClientService methodName: Get size: 116 connection: 172.31.14.131:35846 deadline: 1705315165573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 07:02:51,882 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestNs:TestTable numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 07:02:51,882 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] ipc.CallRunner(144): callId: 237 service: ClientService methodName: Get size: 116 connection: 172.31.14.131:35846 deadline: 1705315165573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 07:02:54,408 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestNs:TestTable numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 07:02:54,408 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] ipc.CallRunner(144): callId: 239 service: ClientService methodName: Get size: 116 connection: 172.31.14.131:35846 deadline: 1705315165573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 07:02:55,079 WARN [hconnection-0x214824e3-shared-pool-10] client.AsyncRequestFutureImpl(819): id=2, table=hbase:quota, attempt=2/46, failureCount=2ops, last exception=org.apache.hadoop.hbase.client.OperationTimeoutExceededException: Timeout exceeded before call began. Meta requests may be slow, the operation timeout is too short for the number of requests, or the configured retries can't complete in the operation timeout. on jenkins-hbase4.apache.org,42073,1702191353745, tracking started Sun Dec 10 07:01:54 UTC 2023; NOT retrying, failed=2 -- final attempt! 2023-12-10 07:02:55,079 WARN [regionserver/jenkins-hbase4:0.Chore.5] quotas.QuotaCache$QuotaRefresherChore(344): Unable to read namespace from quota table org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed 2 actions: Timeout exceeded before call began. Meta requests may be slow, the operation timeout is too short for the number of requests, or the configured retries can't complete in the operation timeout.: 2 times, servers with issues: jenkins-hbase4.apache.org,42073,1702191353745 at org.apache.hadoop.hbase.client.BatchErrors.makeException(BatchErrors.java:50) at org.apache.hadoop.hbase.client.AsyncRequestFutureImpl.getErrors(AsyncRequestFutureImpl.java:1236) at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:434) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:389) at org.apache.hadoop.hbase.quotas.QuotaTableUtil.doGet(QuotaTableUtil.java:910) at org.apache.hadoop.hbase.quotas.QuotaUtil.fetchGlobalQuotas(QuotaUtil.java:374) at org.apache.hadoop.hbase.quotas.QuotaUtil.fetchNamespaceQuotas(QuotaUtil.java:341) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore$1.fetchEntries(QuotaCache.java:242) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetch(QuotaCache.java:333) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.fetchNamespaceQuotaState(QuotaCache.java:234) at org.apache.hadoop.hbase.quotas.QuotaCache$QuotaRefresherChore.chore(QuotaCache.java:225) at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:158) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:107) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2023-12-10 07:02:59,421 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestNs:TestTable numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 07:02:59,422 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37997] ipc.CallRunner(144): callId: 241 service: ClientService methodName: Get size: 116 connection: 172.31.14.131:35846 deadline: 1705315165573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms 2023-12-10 07:02:59,423 DEBUG [Listener at localhost/38369] client.RpcRetryingCallerImpl(129): Call exception, tries=6, retries=16, started=0 ms ago, cancelled=false, msg=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) , details=row 'row-0' on table 'TestNs:TestTable' at region=TestNs:TestTable,1,1702191359393.79758411fb41135ec4572c165c4e2dc5., hostname=jenkins-hbase4.apache.org,37997,1702191353702, seqNum=2, see https://s.apache.org/timeout, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.GeneratedConstructorAccessor45.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:276) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:261) at org.apache.hadoop.hbase.client.RegionServerCallable.call(RegionServerCallable.java:126) at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:104) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:370) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:343) at org.apache.hadoop.hbase.quotas.ThrottleQuotaTestUtil.doGets(ThrottleQuotaTestUtil.java:95) at org.apache.hadoop.hbase.quotas.TestClusterScopeQuotaThrottle.testTableClusterScopeQuota(TestClusterScopeQuotaThrottle.java:151) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.quotas.RpcThrottlingException): org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:381) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:87) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:415) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:411) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:193) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:214) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-12-10 07:02:59,423 ERROR [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(100): get failed after nRetries=0 java.net.SocketTimeoutException: callTimeout=10000, callDuration=10015: org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) row 'row-0' on table 'TestNs:TestTable' at region=TestNs:TestTable,1,1702191359393.79758411fb41135ec4572c165c4e2dc5., hostname=jenkins-hbase4.apache.org,37997,1702191353702, seqNum=2 at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:156) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:370) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:343) at org.apache.hadoop.hbase.quotas.ThrottleQuotaTestUtil.doGets(ThrottleQuotaTestUtil.java:95) at org.apache.hadoop.hbase.quotas.TestClusterScopeQuotaThrottle.testTableClusterScopeQuota(TestClusterScopeQuotaThrottle.java:151) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.quotas.RpcThrottlingException: org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.GeneratedConstructorAccessor45.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:276) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:261) at org.apache.hadoop.hbase.client.RegionServerCallable.call(RegionServerCallable.java:126) at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:104) ... 29 more Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.quotas.RpcThrottlingException): org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 6mins, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:381) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:87) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:415) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:411) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:193) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:214) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-12-10 07:02:59,677 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(184): QuotaCache 2023-12-10 07:02:59,677 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(185): {default=QuotaState(ts=1705318755573 bypass), TestNs=QuotaState(ts=1705318755573 bypass)} 2023-12-10 07:02:59,677 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(186): {TestNs:TestTable=QuotaState(ts=1705318755573 bypass), TestQuotaAdmin1=QuotaState(ts=1705318755573 bypass)} 2023-12-10 07:02:59,677 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(187): {jenkins=UserQuotaState(ts=1705318755573 bypass)} 2023-12-10 07:02:59,677 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(188): {all=QuotaState(ts=1705318755573 bypass)} 2023-12-10 07:02:59,746 INFO [regionserver/jenkins-hbase4:0.Chore.3] regionserver.HRegionServer$PeriodicMemStoreFlusher(1921): MemstoreFlusherChore requesting flush of hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b. because dca2e6ecbeef49fb8929fd36dd74df8b/q has an old edit so flush to free WALs after random delay 233855 ms 2023-12-10 07:02:59,927 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(184): QuotaCache 2023-12-10 07:02:59,928 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(185): {default=QuotaState(ts=1705318755573 bypass), TestNs=QuotaState(ts=1705318755573 bypass)} 2023-12-10 07:02:59,928 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(186): {TestQuotaAdmin0=QuotaState(ts=1705318755573 bypass), TestNs:TestTable=QuotaState(ts=1705318755573 bypass), TestQuotaAdmin2=QuotaState(ts=1705318755573 bypass)} 2023-12-10 07:02:59,928 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(187): {} 2023-12-10 07:02:59,928 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(188): {all=QuotaState(ts=1705318755573 bypass)} 2023-12-10 07:02:59,939 INFO [Listener at localhost/38369] hbase.ResourceChecker(175): after: quotas.TestClusterScopeQuotaThrottle#testTableClusterScopeQuota Thread=285 (was 289), OpenFileDescriptor=594 (was 612), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=89 (was 125), ProcessCount=168 (was 168), AvailableMemoryMB=5061 (was 5051) - AvailableMemoryMB LEAK? - 2023-12-10 07:02:59,950 INFO [Listener at localhost/38369] hbase.ResourceChecker(147): before: quotas.TestClusterScopeQuotaThrottle#testNamespaceClusterScopeQuota Thread=285, OpenFileDescriptor=594, MaxFileDescriptor=60000, SystemLoadAverage=89, ProcessCount=168, AvailableMemoryMB=5060 2023-12-10 07:02:59,970 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2489): Flush column family: q of dca2e6ecbeef49fb8929fd36dd74df8b because time of oldest edit=1705315155573 is > 3600000 from now =1705322355573 2023-12-10 07:02:59,970 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2489): Flush column family: u of dca2e6ecbeef49fb8929fd36dd74df8b because time of oldest edit=1705315155573 is > 3600000 from now =1705322355573 2023-12-10 07:02:59,970 INFO [MemStoreFlusher.0] regionserver.HRegion(2745): Flushing dca2e6ecbeef49fb8929fd36dd74df8b 2/2 column families, dataSize=176 B heapSize=984 B 2023-12-10 07:02:59,986 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=137 B at sequenceid=34 (bloomFilter=true), to=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/.tmp/q/326f869feaac43708b7e34c661209c03 2023-12-10 07:02:59,993 INFO [MemStoreFlusher.0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 326f869feaac43708b7e34c661209c03 2023-12-10 07:03:00,004 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=39 B at sequenceid=34 (bloomFilter=true), to=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/.tmp/u/8c49ee6646a445feb4e6fe7bbd2f3e02 2023-12-10 07:03:00,010 INFO [MemStoreFlusher.0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8c49ee6646a445feb4e6fe7bbd2f3e02 2023-12-10 07:03:00,011 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/.tmp/q/326f869feaac43708b7e34c661209c03 as hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/q/326f869feaac43708b7e34c661209c03 2023-12-10 07:03:00,017 INFO [MemStoreFlusher.0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 326f869feaac43708b7e34c661209c03 2023-12-10 07:03:00,017 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/q/326f869feaac43708b7e34c661209c03, entries=2, sequenceid=34, filesize=5.0 K 2023-12-10 07:03:00,018 DEBUG [MemStoreFlusher.0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/.tmp/u/8c49ee6646a445feb4e6fe7bbd2f3e02 as hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/u/8c49ee6646a445feb4e6fe7bbd2f3e02 2023-12-10 07:03:00,025 INFO [MemStoreFlusher.0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8c49ee6646a445feb4e6fe7bbd2f3e02 2023-12-10 07:03:00,025 INFO [MemStoreFlusher.0] regionserver.HStore(1080): Added hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/u/8c49ee6646a445feb4e6fe7bbd2f3e02, entries=1, sequenceid=34, filesize=5.0 K 2023-12-10 07:03:00,026 INFO [MemStoreFlusher.0] regionserver.HRegion(2948): Finished flush of dataSize ~176 B/176, heapSize ~952 B/952, currentSize=0 B/0 for dca2e6ecbeef49fb8929fd36dd74df8b in 0ms, sequenceid=34, compaction requested=false 2023-12-10 07:03:00,026 DEBUG [MemStoreFlusher.0] regionserver.HRegion(2446): Flush status journal for dca2e6ecbeef49fb8929fd36dd74df8b: 2023-12-10 07:03:00,206 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(184): QuotaCache 2023-12-10 07:03:00,207 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(185): {default=QuotaState(ts=1705322355573 TimeBasedLimiter( writeReqs=AverageIntervalRateLimiter(avail=5 limit=5 tunit=60000) readReqs=AverageIntervalRateLimiter(avail=6 limit=6 tunit=60000))), TestNs=QuotaState(ts=1705322355573 bypass)} 2023-12-10 07:03:00,208 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(186): {TestQuotaAdmin0=QuotaState(ts=0 bypass), TestNs:TestTable=QuotaState(ts=1705322355573 bypass), TestQuotaAdmin1=QuotaState(ts=1705322355573 bypass)} 2023-12-10 07:03:00,208 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(187): {} 2023-12-10 07:03:00,208 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(188): {all=QuotaState(ts=1705322355573 bypass)} 2023-12-10 07:03:00,459 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(184): QuotaCache 2023-12-10 07:03:00,460 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(185): {default=QuotaState(ts=1705322355573 TimeBasedLimiter( writeReqs=AverageIntervalRateLimiter(avail=5 limit=5 tunit=60000) readReqs=AverageIntervalRateLimiter(avail=6 limit=6 tunit=60000))), TestNs=QuotaState(ts=1705322355573 bypass)} 2023-12-10 07:03:00,461 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(186): {TestQuotaAdmin0=QuotaState(ts=1705322355573 bypass), TestNs:TestTable=QuotaState(ts=1705322355573 bypass), TestQuotaAdmin2=QuotaState(ts=1705322355573 bypass)} 2023-12-10 07:03:00,461 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(187): {} 2023-12-10 07:03:00,461 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(188): {all=QuotaState(ts=1705322355573 bypass)} 2023-12-10 07:03:00,468 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=1 numReads=0 numScans=0: number of write requests exceeded - wait 12sec, 0ms 2023-12-10 07:03:00,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 253 service: ClientService methodName: Mutate size: 142 connection: 172.31.14.131:44840 deadline: 1705322365573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 12sec, 0ms 2023-12-10 07:03:00,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=1 numReads=0 numScans=0: number of write requests exceeded - wait 12sec, 0ms 2023-12-10 07:03:00,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 255 service: ClientService methodName: Mutate size: 142 connection: 172.31.14.131:44840 deadline: 1705322365573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 12sec, 0ms 2023-12-10 07:03:01,226 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=1 numReads=0 numScans=0: number of write requests exceeded - wait 12sec, 0ms 2023-12-10 07:03:01,227 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 257 service: ClientService methodName: Mutate size: 142 connection: 172.31.14.131:44840 deadline: 1705322365573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 12sec, 0ms 2023-12-10 07:03:01,980 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=1 numReads=0 numScans=0: number of write requests exceeded - wait 12sec, 0ms 2023-12-10 07:03:01,980 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 259 service: ClientService methodName: Mutate size: 142 connection: 172.31.14.131:44840 deadline: 1705322365573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 12sec, 0ms 2023-12-10 07:03:03,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=1 numReads=0 numScans=0: number of write requests exceeded - wait 12sec, 0ms 2023-12-10 07:03:03,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 261 service: ClientService methodName: Mutate size: 142 connection: 172.31.14.131:44840 deadline: 1705322365573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 12sec, 0ms 2023-12-10 07:03:05,767 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=1 numReads=0 numScans=0: number of write requests exceeded - wait 12sec, 0ms 2023-12-10 07:03:05,767 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 263 service: ClientService methodName: Mutate size: 142 connection: 172.31.14.131:44840 deadline: 1705322365573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 12sec, 0ms 2023-12-10 07:03:10,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=1 numReads=0 numScans=0: number of write requests exceeded - wait 12sec, 0ms 2023-12-10 07:03:10,814 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 265 service: ClientService methodName: Mutate size: 142 connection: 172.31.14.131:44840 deadline: 1705322365573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 12sec, 0ms 2023-12-10 07:03:10,815 DEBUG [Listener at localhost/38369] client.RpcRetryingCallerImpl(129): Call exception, tries=6, retries=16, started=0 ms ago, cancelled=false, msg=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 12sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumWriteRequestsExceeded(RpcThrottlingException.java:104) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:158) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:170) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2942) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) , details=row 'row-5' on table 'TestQuotaAdmin0' at region=TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37., hostname=jenkins-hbase4.apache.org,42073,1702191353745, seqNum=2, see https://s.apache.org/timeout, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 12sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumWriteRequestsExceeded(RpcThrottlingException.java:104) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:158) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:170) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2942) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.GeneratedConstructorAccessor45.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:276) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:261) at org.apache.hadoop.hbase.client.RegionServerCallable.call(RegionServerCallable.java:126) at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:104) at org.apache.hadoop.hbase.client.HTable.put(HTable.java:512) at org.apache.hadoop.hbase.quotas.ThrottleQuotaTestUtil.doPuts(ThrottleQuotaTestUtil.java:71) at org.apache.hadoop.hbase.quotas.ThrottleQuotaTestUtil.doPuts(ThrottleQuotaTestUtil.java:54) at org.apache.hadoop.hbase.quotas.TestClusterScopeQuotaThrottle.testNamespaceClusterScopeQuota(TestClusterScopeQuotaThrottle.java:128) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.quotas.RpcThrottlingException): org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 12sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumWriteRequestsExceeded(RpcThrottlingException.java:104) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:158) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:170) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2942) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:381) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:87) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:415) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:411) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:193) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:214) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-12-10 07:03:10,815 ERROR [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(76): put failed after nRetries=5 java.net.SocketTimeoutException: callTimeout=10000, callDuration=10016: org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 12sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumWriteRequestsExceeded(RpcThrottlingException.java:104) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:158) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:170) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2942) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) row 'row-5' on table 'TestQuotaAdmin0' at region=TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37., hostname=jenkins-hbase4.apache.org,42073,1702191353745, seqNum=2 at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:156) at org.apache.hadoop.hbase.client.HTable.put(HTable.java:512) at org.apache.hadoop.hbase.quotas.ThrottleQuotaTestUtil.doPuts(ThrottleQuotaTestUtil.java:71) at org.apache.hadoop.hbase.quotas.ThrottleQuotaTestUtil.doPuts(ThrottleQuotaTestUtil.java:54) at org.apache.hadoop.hbase.quotas.TestClusterScopeQuotaThrottle.testNamespaceClusterScopeQuota(TestClusterScopeQuotaThrottle.java:128) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.quotas.RpcThrottlingException: org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 12sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumWriteRequestsExceeded(RpcThrottlingException.java:104) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:158) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:170) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2942) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.GeneratedConstructorAccessor45.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:276) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:261) at org.apache.hadoop.hbase.client.RegionServerCallable.call(RegionServerCallable.java:126) at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:104) ... 29 more Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.quotas.RpcThrottlingException): org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of write requests exceeded - wait 12sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumWriteRequestsExceeded(RpcThrottlingException.java:104) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:158) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:170) at org.apache.hadoop.hbase.regionserver.RSRpcServices.mutate(RSRpcServices.java:2942) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44994) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:381) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:87) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:415) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:411) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:193) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:214) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-12-10 07:03:10,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 10sec, 0ms 2023-12-10 07:03:10,820 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 272 service: ClientService methodName: Get size: 114 connection: 172.31.14.131:44840 deadline: 1705322365573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 10sec, 0ms 2023-12-10 07:03:11,072 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 10sec, 0ms 2023-12-10 07:03:11,073 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 274 service: ClientService methodName: Get size: 114 connection: 172.31.14.131:44840 deadline: 1705322365573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 10sec, 0ms 2023-12-10 07:03:11,578 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 10sec, 0ms 2023-12-10 07:03:11,578 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 276 service: ClientService methodName: Get size: 114 connection: 172.31.14.131:44840 deadline: 1705322365573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 10sec, 0ms 2023-12-10 07:03:12,336 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 10sec, 0ms 2023-12-10 07:03:12,336 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 278 service: ClientService methodName: Get size: 114 connection: 172.31.14.131:44840 deadline: 1705322365573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 10sec, 0ms 2023-12-10 07:03:13,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 10sec, 0ms 2023-12-10 07:03:13,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 280 service: ClientService methodName: Get size: 114 connection: 172.31.14.131:44840 deadline: 1705322365573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 10sec, 0ms 2023-12-10 07:03:16,097 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 10sec, 0ms 2023-12-10 07:03:16,097 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 282 service: ClientService methodName: Get size: 114 connection: 172.31.14.131:44840 deadline: 1705322365573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 10sec, 0ms 2023-12-10 07:03:21,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] quotas.RegionServerRpcQuotaManager(222): Throttling exception for user=jenkins table=TestQuotaAdmin0 numWrites=0 numReads=1 numScans=0: number of read requests exceeded - wait 10sec, 0ms 2023-12-10 07:03:21,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=42073] ipc.CallRunner(144): callId: 284 service: ClientService methodName: Get size: 114 connection: 172.31.14.131:44840 deadline: 1705322365573, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 10sec, 0ms 2023-12-10 07:03:21,136 DEBUG [Listener at localhost/38369] client.RpcRetryingCallerImpl(129): Call exception, tries=6, retries=16, started=0 ms ago, cancelled=false, msg=org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 10sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) , details=row 'row-6' on table 'TestQuotaAdmin0' at region=TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37., hostname=jenkins-hbase4.apache.org,42073,1702191353745, seqNum=2, see https://s.apache.org/timeout, exception=org.apache.hadoop.hbase.quotas.RpcThrottlingException: org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 10sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.GeneratedConstructorAccessor45.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:276) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:261) at org.apache.hadoop.hbase.client.RegionServerCallable.call(RegionServerCallable.java:126) at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:104) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:370) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:343) at org.apache.hadoop.hbase.quotas.ThrottleQuotaTestUtil.doGets(ThrottleQuotaTestUtil.java:95) at org.apache.hadoop.hbase.quotas.TestClusterScopeQuotaThrottle.testNamespaceClusterScopeQuota(TestClusterScopeQuotaThrottle.java:129) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.quotas.RpcThrottlingException): org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 10sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:381) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:87) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:415) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:411) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:193) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:214) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-12-10 07:03:21,136 ERROR [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(100): get failed after nRetries=6 java.net.SocketTimeoutException: callTimeout=10000, callDuration=10027: org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 10sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) row 'row-6' on table 'TestQuotaAdmin0' at region=TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37., hostname=jenkins-hbase4.apache.org,42073,1702191353745, seqNum=2 at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:156) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:370) at org.apache.hadoop.hbase.client.HTable.get(HTable.java:343) at org.apache.hadoop.hbase.quotas.ThrottleQuotaTestUtil.doGets(ThrottleQuotaTestUtil.java:95) at org.apache.hadoop.hbase.quotas.TestClusterScopeQuotaThrottle.testNamespaceClusterScopeQuota(TestClusterScopeQuotaThrottle.java:129) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.quotas.RpcThrottlingException: org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 10sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.GeneratedConstructorAccessor45.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:276) at org.apache.hadoop.hbase.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:261) at org.apache.hadoop.hbase.client.RegionServerCallable.call(RegionServerCallable.java:126) at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:104) ... 29 more Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.quotas.RpcThrottlingException): org.apache.hadoop.hbase.quotas.RpcThrottlingException: number of read requests exceeded - wait 10sec, 0ms at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwThrottlingException(RpcThrottlingException.java:133) at org.apache.hadoop.hbase.quotas.RpcThrottlingException.throwNumReadRequestsExceeded(RpcThrottlingException.java:99) at org.apache.hadoop.hbase.quotas.TimeBasedLimiter.checkQuota(TimeBasedLimiter.java:172) at org.apache.hadoop.hbase.quotas.DefaultOperationQuota.checkQuota(DefaultOperationQuota.java:82) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:220) at org.apache.hadoop.hbase.quotas.RegionServerRpcQuotaManager.checkQuota(RegionServerRpcQuotaManager.java:168) at org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:2532) at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:44992) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:381) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:87) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:415) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:411) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:193) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:214) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-12-10 07:03:21,390 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(184): QuotaCache 2023-12-10 07:03:21,390 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(185): {default=QuotaState(ts=1705325955573 bypass), TestNs=QuotaState(ts=1705325955573 bypass)} 2023-12-10 07:03:21,390 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(186): {TestQuotaAdmin0=QuotaState(ts=1705325955573 bypass), TestNs:TestTable=QuotaState(ts=1705325955573 bypass), TestQuotaAdmin1=QuotaState(ts=1705325955573 bypass)} 2023-12-10 07:03:21,390 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(187): {} 2023-12-10 07:03:21,391 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(188): {all=QuotaState(ts=1705325955573 bypass)} 2023-12-10 07:03:21,641 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(184): QuotaCache 2023-12-10 07:03:21,641 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(185): {default=QuotaState(ts=1705325955573 bypass), TestNs=QuotaState(ts=1705325955573 bypass)} 2023-12-10 07:03:21,641 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(186): {TestQuotaAdmin0=QuotaState(ts=1705325955573 bypass), TestNs:TestTable=QuotaState(ts=1705325955573 bypass), TestQuotaAdmin2=QuotaState(ts=1705325955573 bypass)} 2023-12-10 07:03:21,641 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(187): {jenkins=UserQuotaState(ts=1705325955573 bypass)} 2023-12-10 07:03:21,641 DEBUG [Listener at localhost/38369] quotas.ThrottleQuotaTestUtil(188): {all=QuotaState(ts=1705325955573 bypass)} 2023-12-10 07:03:21,653 INFO [Listener at localhost/38369] hbase.ResourceChecker(175): after: quotas.TestClusterScopeQuotaThrottle#testNamespaceClusterScopeQuota Thread=282 (was 285), OpenFileDescriptor=591 (was 594), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=59 (was 89), ProcessCount=166 (was 168), AvailableMemoryMB=5061 (was 5060) - AvailableMemoryMB LEAK? - 2023-12-10 07:03:21,655 INFO [Listener at localhost/38369] client.HBaseAdmin$15(890): Started disable of TestQuotaAdmin0 2023-12-10 07:03:21,659 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.HMaster$11(2424): Client=jenkins//172.31.14.131 disable TestQuotaAdmin0 2023-12-10 07:03:21,666 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] procedure2.ProcedureExecutor(1075): Stored pid=36, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=TestQuotaAdmin0 2023-12-10 07:03:21,669 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestQuotaAdmin0","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1702191801669"}]},"ts":"1702191801669"} 2023-12-10 07:03:21,670 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.MasterRpcServices(1230): Checking to see if procedure is done pid=36 2023-12-10 07:03:21,670 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41596, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-12-10 07:03:21,671 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=TestQuotaAdmin0, state=DISABLING in hbase:meta 2023-12-10 07:03:21,673 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set TestQuotaAdmin0 to state=DISABLING 2023-12-10 07:03:21,675 INFO [PEWorker-1] procedure2.ProcedureExecutor(1796): Initialized subprocedures=[{pid=37, ppid=36, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=TestQuotaAdmin0, region=738cc1ffb58c4f3e8fbde456163cdb37, UNASSIGN}] 2023-12-10 07:03:21,676 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=36, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=TestQuotaAdmin0, region=738cc1ffb58c4f3e8fbde456163cdb37, UNASSIGN 2023-12-10 07:03:21,677 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=738cc1ffb58c4f3e8fbde456163cdb37, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42073,1702191353745 2023-12-10 07:03:21,677 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1702191801677"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1702191801677"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1702191801677"}]},"ts":"1702191801677"} 2023-12-10 07:03:21,678 INFO [PEWorker-2] procedure2.ProcedureExecutor(1796): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure 738cc1ffb58c4f3e8fbde456163cdb37, server=jenkins-hbase4.apache.org,42073,1702191353745}] 2023-12-10 07:03:21,830 DEBUG [RSProcedureDispatcher-pool-6] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,42073,1702191353745 2023-12-10 07:03:21,832 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55670, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-12-10 07:03:21,832 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 738cc1ffb58c4f3e8fbde456163cdb37 2023-12-10 07:03:21,832 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 738cc1ffb58c4f3e8fbde456163cdb37, disabling compactions & flushes 2023-12-10 07:03:21,832 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37. 2023-12-10 07:03:21,832 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37. 2023-12-10 07:03:21,833 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37. after waiting 0 ms 2023-12-10 07:03:21,833 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37. 2023-12-10 07:03:21,833 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 738cc1ffb58c4f3e8fbde456163cdb37 1/1 column families, dataSize=170 B heapSize=816 B 2023-12-10 07:03:21,847 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=170 B at sequenceid=27 (bloomFilter=false), to=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin0/738cc1ffb58c4f3e8fbde456163cdb37/.tmp/cf/b784e52d74c54b2a8680b75262ef7e81 2023-12-10 07:03:21,857 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin0/738cc1ffb58c4f3e8fbde456163cdb37/.tmp/cf/b784e52d74c54b2a8680b75262ef7e81 as hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin0/738cc1ffb58c4f3e8fbde456163cdb37/cf/b784e52d74c54b2a8680b75262ef7e81 2023-12-10 07:03:21,863 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin0/738cc1ffb58c4f3e8fbde456163cdb37/cf/b784e52d74c54b2a8680b75262ef7e81, entries=5, sequenceid=27, filesize=4.7 K 2023-12-10 07:03:21,863 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~170 B/170, heapSize ~800 B/800, currentSize=0 B/0 for 738cc1ffb58c4f3e8fbde456163cdb37 in 30ms, sequenceid=27, compaction requested=false 2023-12-10 07:03:21,869 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin0/738cc1ffb58c4f3e8fbde456163cdb37/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=4 2023-12-10 07:03:21,870 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37. 2023-12-10 07:03:21,870 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 738cc1ffb58c4f3e8fbde456163cdb37: 2023-12-10 07:03:21,871 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 738cc1ffb58c4f3e8fbde456163cdb37 2023-12-10 07:03:21,872 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=738cc1ffb58c4f3e8fbde456163cdb37, regionState=CLOSED 2023-12-10 07:03:21,872 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1702191801872"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1702191801872"}]},"ts":"1702191801872"} 2023-12-10 07:03:21,875 INFO [PEWorker-4] procedure2.ProcedureExecutor(1953): Finished subprocedure pid=38, resume processing ppid=37 2023-12-10 07:03:21,875 INFO [PEWorker-4] procedure2.ProcedureExecutor(1457): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure 738cc1ffb58c4f3e8fbde456163cdb37, server=jenkins-hbase4.apache.org,42073,1702191353745 in 195 msec 2023-12-10 07:03:21,877 INFO [PEWorker-5] procedure2.ProcedureExecutor(1953): Finished subprocedure pid=37, resume processing ppid=36 2023-12-10 07:03:21,877 INFO [PEWorker-5] procedure2.ProcedureExecutor(1457): Finished pid=37, ppid=36, state=SUCCESS; TransitRegionStateProcedure table=TestQuotaAdmin0, region=738cc1ffb58c4f3e8fbde456163cdb37, UNASSIGN in 200 msec 2023-12-10 07:03:21,877 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestQuotaAdmin0","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1702191801877"}]},"ts":"1702191801877"} 2023-12-10 07:03:21,878 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=TestQuotaAdmin0, state=DISABLED in hbase:meta 2023-12-10 07:03:21,880 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set TestQuotaAdmin0 to state=DISABLED 2023-12-10 07:03:21,882 INFO [PEWorker-1] procedure2.ProcedureExecutor(1457): Finished pid=36, state=SUCCESS; DisableTableProcedure table=TestQuotaAdmin0 in 220 msec 2023-12-10 07:03:21,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.MasterRpcServices(1230): Checking to see if procedure is done pid=36 2023-12-10 07:03:21,921 INFO [Listener at localhost/38369] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:TestQuotaAdmin0, procId: 36 completed 2023-12-10 07:03:21,925 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.HMaster$5(2234): Client=jenkins//172.31.14.131 delete TestQuotaAdmin0 2023-12-10 07:03:21,930 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] procedure2.ProcedureExecutor(1075): Stored pid=39, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=TestQuotaAdmin0 2023-12-10 07:03:21,932 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=39, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=TestQuotaAdmin0 2023-12-10 07:03:21,934 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=39, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=TestQuotaAdmin0 2023-12-10 07:03:21,937 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.MasterRpcServices(1230): Checking to see if procedure is done pid=39 2023-12-10 07:03:21,939 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/default/TestQuotaAdmin0/738cc1ffb58c4f3e8fbde456163cdb37 2023-12-10 07:03:21,942 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/default/TestQuotaAdmin0/738cc1ffb58c4f3e8fbde456163cdb37/cf, FileablePath, hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/default/TestQuotaAdmin0/738cc1ffb58c4f3e8fbde456163cdb37/recovered.edits] 2023-12-10 07:03:21,949 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/default/TestQuotaAdmin0/738cc1ffb58c4f3e8fbde456163cdb37/cf/245239aeae9b492dba637fad4f9883c5 to hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/archive/data/default/TestQuotaAdmin0/738cc1ffb58c4f3e8fbde456163cdb37/cf/245239aeae9b492dba637fad4f9883c5 2023-12-10 07:03:21,951 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/default/TestQuotaAdmin0/738cc1ffb58c4f3e8fbde456163cdb37/cf/b784e52d74c54b2a8680b75262ef7e81 to hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/archive/data/default/TestQuotaAdmin0/738cc1ffb58c4f3e8fbde456163cdb37/cf/b784e52d74c54b2a8680b75262ef7e81 2023-12-10 07:03:21,954 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/default/TestQuotaAdmin0/738cc1ffb58c4f3e8fbde456163cdb37/recovered.edits/30.seqid to hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/archive/data/default/TestQuotaAdmin0/738cc1ffb58c4f3e8fbde456163cdb37/recovered.edits/30.seqid 2023-12-10 07:03:21,955 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/default/TestQuotaAdmin0/738cc1ffb58c4f3e8fbde456163cdb37 2023-12-10 07:03:21,955 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived TestQuotaAdmin0 regions 2023-12-10 07:03:21,958 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=39, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=TestQuotaAdmin0 2023-12-10 07:03:21,964 WARN [PEWorker-2] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of TestQuotaAdmin0 from hbase:meta 2023-12-10 07:03:21,966 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(421): Removing 'TestQuotaAdmin0' descriptor. 2023-12-10 07:03:21,967 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=39, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=TestQuotaAdmin0 2023-12-10 07:03:21,967 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(411): Removing 'TestQuotaAdmin0' from region states. 2023-12-10 07:03:21,967 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1702191801967"}]},"ts":"9223372036854775807"} 2023-12-10 07:03:21,969 INFO [PEWorker-2] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-12-10 07:03:21,969 DEBUG [PEWorker-2] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 738cc1ffb58c4f3e8fbde456163cdb37, NAME => 'TestQuotaAdmin0,,1702191356781.738cc1ffb58c4f3e8fbde456163cdb37.', STARTKEY => '', ENDKEY => ''}] 2023-12-10 07:03:21,969 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(415): Marking 'TestQuotaAdmin0' as deleted. 2023-12-10 07:03:21,969 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"TestQuotaAdmin0","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1702191801969"}]},"ts":"9223372036854775807"} 2023-12-10 07:03:21,971 INFO [PEWorker-2] hbase.MetaTableAccessor(1658): Deleted table TestQuotaAdmin0 state from META 2023-12-10 07:03:21,974 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(130): Finished pid=39, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=TestQuotaAdmin0 2023-12-10 07:03:21,976 INFO [PEWorker-2] procedure2.ProcedureExecutor(1457): Finished pid=39, state=SUCCESS; DeleteTableProcedure table=TestQuotaAdmin0 in 49 msec 2023-12-10 07:03:22,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.MasterRpcServices(1230): Checking to see if procedure is done pid=39 2023-12-10 07:03:22,189 INFO [Listener at localhost/38369] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:TestQuotaAdmin0, procId: 39 completed 2023-12-10 07:03:22,189 INFO [Listener at localhost/38369] client.HBaseAdmin$15(890): Started disable of TestQuotaAdmin1 2023-12-10 07:03:22,189 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.HMaster$11(2424): Client=jenkins//172.31.14.131 disable TestQuotaAdmin1 2023-12-10 07:03:22,190 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] procedure2.ProcedureExecutor(1075): Stored pid=40, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=TestQuotaAdmin1 2023-12-10 07:03:22,193 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.MasterRpcServices(1230): Checking to see if procedure is done pid=40 2023-12-10 07:03:22,193 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestQuotaAdmin1","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1702191802193"}]},"ts":"1702191802193"} 2023-12-10 07:03:22,194 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=TestQuotaAdmin1, state=DISABLING in hbase:meta 2023-12-10 07:03:22,196 INFO [PEWorker-3] procedure.DisableTableProcedure(293): Set TestQuotaAdmin1 to state=DISABLING 2023-12-10 07:03:22,197 INFO [PEWorker-3] procedure2.ProcedureExecutor(1796): Initialized subprocedures=[{pid=41, ppid=40, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=TestQuotaAdmin1, region=0522cf92a3f0ad796f58de4381345d12, UNASSIGN}] 2023-12-10 07:03:22,198 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=41, ppid=40, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=TestQuotaAdmin1, region=0522cf92a3f0ad796f58de4381345d12, UNASSIGN 2023-12-10 07:03:22,198 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=41 updating hbase:meta row=0522cf92a3f0ad796f58de4381345d12, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 07:03:22,198 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestQuotaAdmin1,,1702191357579.0522cf92a3f0ad796f58de4381345d12.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1702191802198"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1702191802198"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1702191802198"}]},"ts":"1702191802198"} 2023-12-10 07:03:22,200 INFO [PEWorker-4] procedure2.ProcedureExecutor(1796): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE; CloseRegionProcedure 0522cf92a3f0ad796f58de4381345d12, server=jenkins-hbase4.apache.org,37997,1702191353702}] 2023-12-10 07:03:22,351 DEBUG [RSProcedureDispatcher-pool-7] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 07:03:22,353 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48224, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-12-10 07:03:22,354 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 0522cf92a3f0ad796f58de4381345d12 2023-12-10 07:03:22,354 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 0522cf92a3f0ad796f58de4381345d12, disabling compactions & flushes 2023-12-10 07:03:22,354 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestQuotaAdmin1,,1702191357579.0522cf92a3f0ad796f58de4381345d12. 2023-12-10 07:03:22,354 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestQuotaAdmin1,,1702191357579.0522cf92a3f0ad796f58de4381345d12. 2023-12-10 07:03:22,354 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestQuotaAdmin1,,1702191357579.0522cf92a3f0ad796f58de4381345d12. after waiting 0 ms 2023-12-10 07:03:22,354 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestQuotaAdmin1,,1702191357579.0522cf92a3f0ad796f58de4381345d12. 2023-12-10 07:03:22,360 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin1/0522cf92a3f0ad796f58de4381345d12/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-12-10 07:03:22,361 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestQuotaAdmin1,,1702191357579.0522cf92a3f0ad796f58de4381345d12. 2023-12-10 07:03:22,361 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 0522cf92a3f0ad796f58de4381345d12: 2023-12-10 07:03:22,362 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 0522cf92a3f0ad796f58de4381345d12 2023-12-10 07:03:22,363 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=41 updating hbase:meta row=0522cf92a3f0ad796f58de4381345d12, regionState=CLOSED 2023-12-10 07:03:22,363 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"TestQuotaAdmin1,,1702191357579.0522cf92a3f0ad796f58de4381345d12.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1702191802363"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1702191802363"}]},"ts":"1702191802363"} 2023-12-10 07:03:22,366 INFO [PEWorker-1] procedure2.ProcedureExecutor(1953): Finished subprocedure pid=42, resume processing ppid=41 2023-12-10 07:03:22,366 INFO [PEWorker-1] procedure2.ProcedureExecutor(1457): Finished pid=42, ppid=41, state=SUCCESS; CloseRegionProcedure 0522cf92a3f0ad796f58de4381345d12, server=jenkins-hbase4.apache.org,37997,1702191353702 in 164 msec 2023-12-10 07:03:22,368 INFO [PEWorker-2] procedure2.ProcedureExecutor(1953): Finished subprocedure pid=41, resume processing ppid=40 2023-12-10 07:03:22,368 INFO [PEWorker-2] procedure2.ProcedureExecutor(1457): Finished pid=41, ppid=40, state=SUCCESS; TransitRegionStateProcedure table=TestQuotaAdmin1, region=0522cf92a3f0ad796f58de4381345d12, UNASSIGN in 169 msec 2023-12-10 07:03:22,368 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestQuotaAdmin1","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1702191802368"}]},"ts":"1702191802368"} 2023-12-10 07:03:22,369 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=TestQuotaAdmin1, state=DISABLED in hbase:meta 2023-12-10 07:03:22,372 INFO [PEWorker-3] procedure.DisableTableProcedure(305): Set TestQuotaAdmin1 to state=DISABLED 2023-12-10 07:03:22,373 INFO [PEWorker-3] procedure2.ProcedureExecutor(1457): Finished pid=40, state=SUCCESS; DisableTableProcedure table=TestQuotaAdmin1 in 183 msec 2023-12-10 07:03:22,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.MasterRpcServices(1230): Checking to see if procedure is done pid=40 2023-12-10 07:03:22,443 INFO [Listener at localhost/38369] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:TestQuotaAdmin1, procId: 40 completed 2023-12-10 07:03:22,444 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.HMaster$5(2234): Client=jenkins//172.31.14.131 delete TestQuotaAdmin1 2023-12-10 07:03:22,445 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] procedure2.ProcedureExecutor(1075): Stored pid=43, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=TestQuotaAdmin1 2023-12-10 07:03:22,446 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=43, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=TestQuotaAdmin1 2023-12-10 07:03:22,447 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=43, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=TestQuotaAdmin1 2023-12-10 07:03:22,448 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.MasterRpcServices(1230): Checking to see if procedure is done pid=43 2023-12-10 07:03:22,451 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/default/TestQuotaAdmin1/0522cf92a3f0ad796f58de4381345d12 2023-12-10 07:03:22,452 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/default/TestQuotaAdmin1/0522cf92a3f0ad796f58de4381345d12/cf, FileablePath, hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/default/TestQuotaAdmin1/0522cf92a3f0ad796f58de4381345d12/recovered.edits] 2023-12-10 07:03:22,458 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/default/TestQuotaAdmin1/0522cf92a3f0ad796f58de4381345d12/recovered.edits/4.seqid to hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/archive/data/default/TestQuotaAdmin1/0522cf92a3f0ad796f58de4381345d12/recovered.edits/4.seqid 2023-12-10 07:03:22,459 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/default/TestQuotaAdmin1/0522cf92a3f0ad796f58de4381345d12 2023-12-10 07:03:22,459 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived TestQuotaAdmin1 regions 2023-12-10 07:03:22,461 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=43, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=TestQuotaAdmin1 2023-12-10 07:03:22,464 WARN [PEWorker-4] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of TestQuotaAdmin1 from hbase:meta 2023-12-10 07:03:22,465 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(421): Removing 'TestQuotaAdmin1' descriptor. 2023-12-10 07:03:22,467 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=43, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=TestQuotaAdmin1 2023-12-10 07:03:22,467 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(411): Removing 'TestQuotaAdmin1' from region states. 2023-12-10 07:03:22,467 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"TestQuotaAdmin1,,1702191357579.0522cf92a3f0ad796f58de4381345d12.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1702191802467"}]},"ts":"9223372036854775807"} 2023-12-10 07:03:22,468 INFO [PEWorker-4] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-12-10 07:03:22,468 DEBUG [PEWorker-4] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 0522cf92a3f0ad796f58de4381345d12, NAME => 'TestQuotaAdmin1,,1702191357579.0522cf92a3f0ad796f58de4381345d12.', STARTKEY => '', ENDKEY => ''}] 2023-12-10 07:03:22,468 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(415): Marking 'TestQuotaAdmin1' as deleted. 2023-12-10 07:03:22,469 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"TestQuotaAdmin1","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1702191802469"}]},"ts":"9223372036854775807"} 2023-12-10 07:03:22,470 INFO [PEWorker-4] hbase.MetaTableAccessor(1658): Deleted table TestQuotaAdmin1 state from META 2023-12-10 07:03:22,472 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(130): Finished pid=43, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=TestQuotaAdmin1 2023-12-10 07:03:22,474 INFO [PEWorker-4] procedure2.ProcedureExecutor(1457): Finished pid=43, state=SUCCESS; DeleteTableProcedure table=TestQuotaAdmin1 in 29 msec 2023-12-10 07:03:22,699 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.MasterRpcServices(1230): Checking to see if procedure is done pid=43 2023-12-10 07:03:22,699 INFO [Listener at localhost/38369] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:TestQuotaAdmin1, procId: 43 completed 2023-12-10 07:03:22,699 INFO [Listener at localhost/38369] client.HBaseAdmin$15(890): Started disable of TestQuotaAdmin2 2023-12-10 07:03:22,700 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.HMaster$11(2424): Client=jenkins//172.31.14.131 disable TestQuotaAdmin2 2023-12-10 07:03:22,701 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] procedure2.ProcedureExecutor(1075): Stored pid=44, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=TestQuotaAdmin2 2023-12-10 07:03:22,704 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.MasterRpcServices(1230): Checking to see if procedure is done pid=44 2023-12-10 07:03:22,705 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestQuotaAdmin2","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1702191802705"}]},"ts":"1702191802705"} 2023-12-10 07:03:22,706 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=TestQuotaAdmin2, state=DISABLING in hbase:meta 2023-12-10 07:03:22,709 INFO [PEWorker-5] procedure.DisableTableProcedure(293): Set TestQuotaAdmin2 to state=DISABLING 2023-12-10 07:03:22,710 INFO [PEWorker-5] procedure2.ProcedureExecutor(1796): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=TestQuotaAdmin2, region=700b1850706b553c6f995417c6bb917e, UNASSIGN}] 2023-12-10 07:03:22,711 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=TestQuotaAdmin2, region=700b1850706b553c6f995417c6bb917e, UNASSIGN 2023-12-10 07:03:22,712 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=700b1850706b553c6f995417c6bb917e, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42073,1702191353745 2023-12-10 07:03:22,712 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1702191802712"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1702191802712"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1702191802712"}]},"ts":"1702191802712"} 2023-12-10 07:03:22,713 INFO [PEWorker-1] procedure2.ProcedureExecutor(1796): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 700b1850706b553c6f995417c6bb917e, server=jenkins-hbase4.apache.org,42073,1702191353745}] 2023-12-10 07:03:22,865 DEBUG [RSProcedureDispatcher-pool-8] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,42073,1702191353745 2023-12-10 07:03:22,866 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 700b1850706b553c6f995417c6bb917e 2023-12-10 07:03:22,866 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 700b1850706b553c6f995417c6bb917e, disabling compactions & flushes 2023-12-10 07:03:22,866 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e. 2023-12-10 07:03:22,866 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e. 2023-12-10 07:03:22,866 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e. after waiting 0 ms 2023-12-10 07:03:22,866 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e. 2023-12-10 07:03:22,870 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/default/TestQuotaAdmin2/700b1850706b553c6f995417c6bb917e/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-12-10 07:03:22,871 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e. 2023-12-10 07:03:22,871 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 700b1850706b553c6f995417c6bb917e: 2023-12-10 07:03:22,873 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 700b1850706b553c6f995417c6bb917e 2023-12-10 07:03:22,873 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=700b1850706b553c6f995417c6bb917e, regionState=CLOSED 2023-12-10 07:03:22,873 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1702191802873"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1702191802873"}]},"ts":"1702191802873"} 2023-12-10 07:03:22,876 INFO [PEWorker-3] procedure2.ProcedureExecutor(1953): Finished subprocedure pid=46, resume processing ppid=45 2023-12-10 07:03:22,876 INFO [PEWorker-3] procedure2.ProcedureExecutor(1457): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 700b1850706b553c6f995417c6bb917e, server=jenkins-hbase4.apache.org,42073,1702191353745 in 162 msec 2023-12-10 07:03:22,878 INFO [PEWorker-4] procedure2.ProcedureExecutor(1953): Finished subprocedure pid=45, resume processing ppid=44 2023-12-10 07:03:22,878 INFO [PEWorker-4] procedure2.ProcedureExecutor(1457): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=TestQuotaAdmin2, region=700b1850706b553c6f995417c6bb917e, UNASSIGN in 166 msec 2023-12-10 07:03:22,878 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestQuotaAdmin2","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1702191802878"}]},"ts":"1702191802878"} 2023-12-10 07:03:22,879 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=TestQuotaAdmin2, state=DISABLED in hbase:meta 2023-12-10 07:03:22,881 INFO [PEWorker-5] procedure.DisableTableProcedure(305): Set TestQuotaAdmin2 to state=DISABLED 2023-12-10 07:03:22,883 INFO [PEWorker-5] procedure2.ProcedureExecutor(1457): Finished pid=44, state=SUCCESS; DisableTableProcedure table=TestQuotaAdmin2 in 182 msec 2023-12-10 07:03:22,954 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.MasterRpcServices(1230): Checking to see if procedure is done pid=44 2023-12-10 07:03:22,955 INFO [Listener at localhost/38369] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:TestQuotaAdmin2, procId: 44 completed 2023-12-10 07:03:22,955 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.HMaster$5(2234): Client=jenkins//172.31.14.131 delete TestQuotaAdmin2 2023-12-10 07:03:22,956 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] procedure2.ProcedureExecutor(1075): Stored pid=47, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=TestQuotaAdmin2 2023-12-10 07:03:22,957 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=47, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=TestQuotaAdmin2 2023-12-10 07:03:22,958 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=47, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=TestQuotaAdmin2 2023-12-10 07:03:22,959 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.MasterRpcServices(1230): Checking to see if procedure is done pid=47 2023-12-10 07:03:22,962 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/default/TestQuotaAdmin2/700b1850706b553c6f995417c6bb917e 2023-12-10 07:03:22,964 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/default/TestQuotaAdmin2/700b1850706b553c6f995417c6bb917e/cf, FileablePath, hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/default/TestQuotaAdmin2/700b1850706b553c6f995417c6bb917e/recovered.edits] 2023-12-10 07:03:22,969 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/default/TestQuotaAdmin2/700b1850706b553c6f995417c6bb917e/recovered.edits/7.seqid to hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/archive/data/default/TestQuotaAdmin2/700b1850706b553c6f995417c6bb917e/recovered.edits/7.seqid 2023-12-10 07:03:22,970 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/default/TestQuotaAdmin2/700b1850706b553c6f995417c6bb917e 2023-12-10 07:03:22,970 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived TestQuotaAdmin2 regions 2023-12-10 07:03:22,972 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=47, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=TestQuotaAdmin2 2023-12-10 07:03:22,974 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of TestQuotaAdmin2 from hbase:meta 2023-12-10 07:03:22,975 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'TestQuotaAdmin2' descriptor. 2023-12-10 07:03:22,976 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=47, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=TestQuotaAdmin2 2023-12-10 07:03:22,976 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'TestQuotaAdmin2' from region states. 2023-12-10 07:03:22,976 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1702191802976"}]},"ts":"9223372036854775807"} 2023-12-10 07:03:22,977 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-12-10 07:03:22,977 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 700b1850706b553c6f995417c6bb917e, NAME => 'TestQuotaAdmin2,,1702191358356.700b1850706b553c6f995417c6bb917e.', STARTKEY => '', ENDKEY => ''}] 2023-12-10 07:03:22,977 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'TestQuotaAdmin2' as deleted. 2023-12-10 07:03:22,978 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"TestQuotaAdmin2","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1702191802978"}]},"ts":"9223372036854775807"} 2023-12-10 07:03:22,979 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table TestQuotaAdmin2 state from META 2023-12-10 07:03:22,981 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=47, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=TestQuotaAdmin2 2023-12-10 07:03:22,982 INFO [PEWorker-1] procedure2.ProcedureExecutor(1457): Finished pid=47, state=SUCCESS; DeleteTableProcedure table=TestQuotaAdmin2 in 26 msec 2023-12-10 07:03:23,210 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.MasterRpcServices(1230): Checking to see if procedure is done pid=47 2023-12-10 07:03:23,210 INFO [Listener at localhost/38369] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:TestQuotaAdmin2, procId: 47 completed 2023-12-10 07:03:23,210 INFO [Listener at localhost/38369] client.HBaseAdmin$15(890): Started disable of TestNs:TestTable 2023-12-10 07:03:23,211 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.HMaster$11(2424): Client=jenkins//172.31.14.131 disable TestNs:TestTable 2023-12-10 07:03:23,211 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] procedure2.ProcedureExecutor(1075): Stored pid=48, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=TestNs:TestTable 2023-12-10 07:03:23,215 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.MasterRpcServices(1230): Checking to see if procedure is done pid=48 2023-12-10 07:03:23,215 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestNs:TestTable","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1702191803215"}]},"ts":"1702191803215"} 2023-12-10 07:03:23,216 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=TestNs:TestTable, state=DISABLING in hbase:meta 2023-12-10 07:03:23,218 INFO [PEWorker-2] procedure.DisableTableProcedure(293): Set TestNs:TestTable to state=DISABLING 2023-12-10 07:03:23,219 INFO [PEWorker-2] procedure2.ProcedureExecutor(1796): Initialized subprocedures=[{pid=49, ppid=48, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=TestNs:TestTable, region=6f85f7a404e69950adc2db5f74a3bc3b, UNASSIGN}, {pid=50, ppid=48, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=TestNs:TestTable, region=79758411fb41135ec4572c165c4e2dc5, UNASSIGN}] 2023-12-10 07:03:23,220 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=49, ppid=48, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=TestNs:TestTable, region=6f85f7a404e69950adc2db5f74a3bc3b, UNASSIGN 2023-12-10 07:03:23,220 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=48, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=TestNs:TestTable, region=79758411fb41135ec4572c165c4e2dc5, UNASSIGN 2023-12-10 07:03:23,221 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=49 updating hbase:meta row=6f85f7a404e69950adc2db5f74a3bc3b, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,42073,1702191353745 2023-12-10 07:03:23,221 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=79758411fb41135ec4572c165c4e2dc5, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 07:03:23,221 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestNs:TestTable,,1702191359393.6f85f7a404e69950adc2db5f74a3bc3b.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1702191803221"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1702191803221"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1702191803221"}]},"ts":"1702191803221"} 2023-12-10 07:03:23,221 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"TestNs:TestTable,1,1702191359393.79758411fb41135ec4572c165c4e2dc5.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1702191803221"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1702191803221"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1702191803221"}]},"ts":"1702191803221"} 2023-12-10 07:03:23,222 INFO [PEWorker-2] procedure2.ProcedureExecutor(1796): Initialized subprocedures=[{pid=51, ppid=49, state=RUNNABLE; CloseRegionProcedure 6f85f7a404e69950adc2db5f74a3bc3b, server=jenkins-hbase4.apache.org,42073,1702191353745}] 2023-12-10 07:03:23,223 INFO [PEWorker-3] procedure2.ProcedureExecutor(1796): Initialized subprocedures=[{pid=52, ppid=50, state=RUNNABLE; CloseRegionProcedure 79758411fb41135ec4572c165c4e2dc5, server=jenkins-hbase4.apache.org,37997,1702191353702}] 2023-12-10 07:03:23,373 DEBUG [RSProcedureDispatcher-pool-6] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,42073,1702191353745 2023-12-10 07:03:23,373 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 6f85f7a404e69950adc2db5f74a3bc3b 2023-12-10 07:03:23,374 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6f85f7a404e69950adc2db5f74a3bc3b, disabling compactions & flushes 2023-12-10 07:03:23,374 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestNs:TestTable,,1702191359393.6f85f7a404e69950adc2db5f74a3bc3b. 2023-12-10 07:03:23,374 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestNs:TestTable,,1702191359393.6f85f7a404e69950adc2db5f74a3bc3b. 2023-12-10 07:03:23,374 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestNs:TestTable,,1702191359393.6f85f7a404e69950adc2db5f74a3bc3b. after waiting 0 ms 2023-12-10 07:03:23,374 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestNs:TestTable,,1702191359393.6f85f7a404e69950adc2db5f74a3bc3b. 2023-12-10 07:03:23,375 DEBUG [RSProcedureDispatcher-pool-7] master.ServerManager(702): New admin connection to jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 07:03:23,375 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 79758411fb41135ec4572c165c4e2dc5 2023-12-10 07:03:23,376 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 79758411fb41135ec4572c165c4e2dc5, disabling compactions & flushes 2023-12-10 07:03:23,376 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region TestNs:TestTable,1,1702191359393.79758411fb41135ec4572c165c4e2dc5. 2023-12-10 07:03:23,376 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on TestNs:TestTable,1,1702191359393.79758411fb41135ec4572c165c4e2dc5. 2023-12-10 07:03:23,376 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on TestNs:TestTable,1,1702191359393.79758411fb41135ec4572c165c4e2dc5. after waiting 0 ms 2023-12-10 07:03:23,376 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region TestNs:TestTable,1,1702191359393.79758411fb41135ec4572c165c4e2dc5. 2023-12-10 07:03:23,378 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/TestNs/TestTable/6f85f7a404e69950adc2db5f74a3bc3b/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-12-10 07:03:23,380 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestNs:TestTable,,1702191359393.6f85f7a404e69950adc2db5f74a3bc3b. 2023-12-10 07:03:23,380 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6f85f7a404e69950adc2db5f74a3bc3b: 2023-12-10 07:03:23,380 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/TestNs/TestTable/79758411fb41135ec4572c165c4e2dc5/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-12-10 07:03:23,381 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 6f85f7a404e69950adc2db5f74a3bc3b 2023-12-10 07:03:23,381 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed TestNs:TestTable,1,1702191359393.79758411fb41135ec4572c165c4e2dc5. 2023-12-10 07:03:23,382 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 79758411fb41135ec4572c165c4e2dc5: 2023-12-10 07:03:23,382 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=49 updating hbase:meta row=6f85f7a404e69950adc2db5f74a3bc3b, regionState=CLOSED 2023-12-10 07:03:23,382 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"TestNs:TestTable,,1702191359393.6f85f7a404e69950adc2db5f74a3bc3b.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1702191803382"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1702191803382"}]},"ts":"1702191803382"} 2023-12-10 07:03:23,383 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 79758411fb41135ec4572c165c4e2dc5 2023-12-10 07:03:23,383 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=79758411fb41135ec4572c165c4e2dc5, regionState=CLOSED 2023-12-10 07:03:23,383 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"TestNs:TestTable,1,1702191359393.79758411fb41135ec4572c165c4e2dc5.","families":{"info":[{"qualifier":"regioninfo","vlen":43,"tag":[],"timestamp":"1702191803383"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1702191803383"}]},"ts":"1702191803383"} 2023-12-10 07:03:23,385 INFO [PEWorker-4] procedure2.ProcedureExecutor(1953): Finished subprocedure pid=51, resume processing ppid=49 2023-12-10 07:03:23,385 INFO [PEWorker-4] procedure2.ProcedureExecutor(1457): Finished pid=51, ppid=49, state=SUCCESS; CloseRegionProcedure 6f85f7a404e69950adc2db5f74a3bc3b, server=jenkins-hbase4.apache.org,42073,1702191353745 in 161 msec 2023-12-10 07:03:23,386 INFO [PEWorker-2] procedure2.ProcedureExecutor(1953): Finished subprocedure pid=52, resume processing ppid=50 2023-12-10 07:03:23,386 INFO [PEWorker-2] procedure2.ProcedureExecutor(1457): Finished pid=52, ppid=50, state=SUCCESS; CloseRegionProcedure 79758411fb41135ec4572c165c4e2dc5, server=jenkins-hbase4.apache.org,37997,1702191353702 in 161 msec 2023-12-10 07:03:23,387 INFO [PEWorker-3] procedure2.ProcedureExecutor(1457): Finished pid=49, ppid=48, state=SUCCESS; TransitRegionStateProcedure table=TestNs:TestTable, region=6f85f7a404e69950adc2db5f74a3bc3b, UNASSIGN in 166 msec 2023-12-10 07:03:23,387 INFO [PEWorker-5] procedure2.ProcedureExecutor(1953): Finished subprocedure pid=50, resume processing ppid=48 2023-12-10 07:03:23,387 INFO [PEWorker-5] procedure2.ProcedureExecutor(1457): Finished pid=50, ppid=48, state=SUCCESS; TransitRegionStateProcedure table=TestNs:TestTable, region=79758411fb41135ec4572c165c4e2dc5, UNASSIGN in 167 msec 2023-12-10 07:03:23,388 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"TestNs:TestTable","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1702191803388"}]},"ts":"1702191803388"} 2023-12-10 07:03:23,389 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=TestNs:TestTable, state=DISABLED in hbase:meta 2023-12-10 07:03:23,391 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set TestNs:TestTable to state=DISABLED 2023-12-10 07:03:23,393 INFO [PEWorker-1] procedure2.ProcedureExecutor(1457): Finished pid=48, state=SUCCESS; DisableTableProcedure table=TestNs:TestTable in 181 msec 2023-12-10 07:03:23,465 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.MasterRpcServices(1230): Checking to see if procedure is done pid=48 2023-12-10 07:03:23,465 INFO [Listener at localhost/38369] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: TestNs:TestTable, procId: 48 completed 2023-12-10 07:03:23,466 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.HMaster$5(2234): Client=jenkins//172.31.14.131 delete TestNs:TestTable 2023-12-10 07:03:23,466 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] procedure2.ProcedureExecutor(1075): Stored pid=53, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=TestNs:TestTable 2023-12-10 07:03:23,468 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=53, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=TestNs:TestTable 2023-12-10 07:03:23,469 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=53, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=TestNs:TestTable 2023-12-10 07:03:23,471 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.MasterRpcServices(1230): Checking to see if procedure is done pid=53 2023-12-10 07:03:23,472 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/TestNs/TestTable/6f85f7a404e69950adc2db5f74a3bc3b 2023-12-10 07:03:23,473 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/TestNs/TestTable/79758411fb41135ec4572c165c4e2dc5 2023-12-10 07:03:23,474 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/TestNs/TestTable/6f85f7a404e69950adc2db5f74a3bc3b/cf, FileablePath, hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/TestNs/TestTable/6f85f7a404e69950adc2db5f74a3bc3b/recovered.edits] 2023-12-10 07:03:23,474 DEBUG [HFileArchiver-12] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/TestNs/TestTable/79758411fb41135ec4572c165c4e2dc5/cf, FileablePath, hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/TestNs/TestTable/79758411fb41135ec4572c165c4e2dc5/recovered.edits] 2023-12-10 07:03:23,480 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/TestNs/TestTable/6f85f7a404e69950adc2db5f74a3bc3b/recovered.edits/4.seqid to hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/archive/data/TestNs/TestTable/6f85f7a404e69950adc2db5f74a3bc3b/recovered.edits/4.seqid 2023-12-10 07:03:23,481 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/TestNs/TestTable/79758411fb41135ec4572c165c4e2dc5/recovered.edits/4.seqid to hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/archive/data/TestNs/TestTable/79758411fb41135ec4572c165c4e2dc5/recovered.edits/4.seqid 2023-12-10 07:03:23,481 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/TestNs/TestTable/6f85f7a404e69950adc2db5f74a3bc3b 2023-12-10 07:03:23,481 DEBUG [HFileArchiver-12] backup.HFileArchiver(596): Deleted hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/.tmp/data/TestNs/TestTable/79758411fb41135ec4572c165c4e2dc5 2023-12-10 07:03:23,482 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived TestNs:TestTable regions 2023-12-10 07:03:23,484 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=53, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=TestNs:TestTable 2023-12-10 07:03:23,486 WARN [PEWorker-4] procedure.DeleteTableProcedure(384): Deleting some vestigial 2 rows of TestNs:TestTable from hbase:meta 2023-12-10 07:03:23,488 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(421): Removing 'TestNs:TestTable' descriptor. 2023-12-10 07:03:23,489 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=53, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=TestNs:TestTable 2023-12-10 07:03:23,489 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(411): Removing 'TestNs:TestTable' from region states. 2023-12-10 07:03:23,489 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"TestNs:TestTable,,1702191359393.6f85f7a404e69950adc2db5f74a3bc3b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1702191803489"}]},"ts":"9223372036854775807"} 2023-12-10 07:03:23,489 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"TestNs:TestTable,1,1702191359393.79758411fb41135ec4572c165c4e2dc5.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1702191803489"}]},"ts":"9223372036854775807"} 2023-12-10 07:03:23,491 INFO [PEWorker-4] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-12-10 07:03:23,491 DEBUG [PEWorker-4] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 6f85f7a404e69950adc2db5f74a3bc3b, NAME => 'TestNs:TestTable,,1702191359393.6f85f7a404e69950adc2db5f74a3bc3b.', STARTKEY => '', ENDKEY => '1'}, {ENCODED => 79758411fb41135ec4572c165c4e2dc5, NAME => 'TestNs:TestTable,1,1702191359393.79758411fb41135ec4572c165c4e2dc5.', STARTKEY => '1', ENDKEY => ''}] 2023-12-10 07:03:23,491 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(415): Marking 'TestNs:TestTable' as deleted. 2023-12-10 07:03:23,491 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"TestNs:TestTable","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1702191803491"}]},"ts":"9223372036854775807"} 2023-12-10 07:03:23,492 INFO [PEWorker-4] hbase.MetaTableAccessor(1658): Deleted table TestNs:TestTable state from META 2023-12-10 07:03:23,495 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(130): Finished pid=53, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=TestNs:TestTable 2023-12-10 07:03:23,496 INFO [PEWorker-4] procedure2.ProcedureExecutor(1457): Finished pid=53, state=SUCCESS; DeleteTableProcedure table=TestNs:TestTable in 29 msec 2023-12-10 07:03:23,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.MasterRpcServices(1230): Checking to see if procedure is done pid=53 2023-12-10 07:03:23,722 INFO [Listener at localhost/38369] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: TestNs:TestTable, procId: 53 completed 2023-12-10 07:03:23,725 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.HMaster$17(3092): Client=jenkins//172.31.14.131 delete TestNs 2023-12-10 07:03:23,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] procedure2.ProcedureExecutor(1075): Stored pid=54, state=RUNNABLE:DELETE_NAMESPACE_PREPARE; DeleteNamespaceProcedure, namespace=TestNs 2023-12-10 07:03:23,733 INFO [PEWorker-2] procedure.DeleteNamespaceProcedure(73): pid=54, state=RUNNABLE:DELETE_NAMESPACE_PREPARE, locked=true; DeleteNamespaceProcedure, namespace=TestNs 2023-12-10 07:03:23,737 INFO [PEWorker-2] procedure.DeleteNamespaceProcedure(73): pid=54, state=RUNNABLE:DELETE_NAMESPACE_DELETE_FROM_NS_TABLE, locked=true; DeleteNamespaceProcedure, namespace=TestNs 2023-12-10 07:03:23,740 INFO [PEWorker-2] procedure.DeleteNamespaceProcedure(73): pid=54, state=RUNNABLE:DELETE_NAMESPACE_REMOVE_FROM_ZK, locked=true; DeleteNamespaceProcedure, namespace=TestNs 2023-12-10 07:03:23,740 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.MasterRpcServices(1230): Checking to see if procedure is done pid=54 2023-12-10 07:03:23,741 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/namespace/TestNs 2023-12-10 07:03:23,741 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-12-10 07:03:23,742 INFO [PEWorker-2] procedure.DeleteNamespaceProcedure(73): pid=54, state=RUNNABLE:DELETE_NAMESPACE_DELETE_DIRECTORIES, locked=true; DeleteNamespaceProcedure, namespace=TestNs 2023-12-10 07:03:23,744 INFO [PEWorker-2] procedure.DeleteNamespaceProcedure(73): pid=54, state=RUNNABLE:DELETE_NAMESPACE_REMOVE_NAMESPACE_QUOTA, locked=true; DeleteNamespaceProcedure, namespace=TestNs 2023-12-10 07:03:23,745 INFO [PEWorker-2] procedure2.ProcedureExecutor(1457): Finished pid=54, state=SUCCESS; DeleteNamespaceProcedure, namespace=TestNs in 18 msec 2023-12-10 07:03:23,990 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=40533] master.MasterRpcServices(1230): Checking to see if procedure is done pid=54 2023-12-10 07:03:23,991 INFO [Listener at localhost/38369] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-12-10 07:03:23,991 INFO [Listener at localhost/38369] client.ConnectionImplementation(1973): Closing master protocol: MasterService 2023-12-10 07:03:23,991 DEBUG [Listener at localhost/38369] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x11b46918 to 127.0.0.1:51670 2023-12-10 07:03:23,991 DEBUG [Listener at localhost/38369] ipc.AbstractRpcClient(489): Stopping rpc client 2023-12-10 07:03:23,991 DEBUG [Listener at localhost/38369] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-12-10 07:03:23,991 DEBUG [Listener at localhost/38369] util.JVMClusterUtil(257): Found active master hash=1723894950, stopped=false 2023-12-10 07:03:23,991 DEBUG [Listener at localhost/38369] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.quotas.MasterQuotasObserver 2023-12-10 07:03:23,992 INFO [Listener at localhost/38369] master.ServerManager(888): Cluster shutdown requested of master=jenkins-hbase4.apache.org,40533,1702191352570 2023-12-10 07:03:23,994 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-12-10 07:03:23,994 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): regionserver:37997-0x1007092abbd0001, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-12-10 07:03:23,994 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-12-10 07:03:23,994 INFO [Listener at localhost/38369] procedure2.ProcedureExecutor(675): Stopping 2023-12-10 07:03:23,994 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): regionserver:42073-0x1007092abbd0002, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-12-10 07:03:23,995 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:37997-0x1007092abbd0001, quorum=127.0.0.1:51670, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-12-10 07:03:23,995 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-12-10 07:03:23,995 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): regionserver:42073-0x1007092abbd0002, quorum=127.0.0.1:51670, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-12-10 07:03:23,995 DEBUG [Listener at localhost/38369] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x61d15571 to 127.0.0.1:51670 2023-12-10 07:03:23,995 DEBUG [Listener at localhost/38369] ipc.AbstractRpcClient(489): Stopping rpc client 2023-12-10 07:03:23,995 INFO [Listener at localhost/38369] regionserver.HRegionServer(2300): ***** STOPPING region server 'jenkins-hbase4.apache.org,37997,1702191353702' ***** 2023-12-10 07:03:23,995 INFO [Listener at localhost/38369] regionserver.HRegionServer(2314): STOPPED: Shutdown requested 2023-12-10 07:03:23,995 INFO [Listener at localhost/38369] regionserver.HRegionServer(2300): ***** STOPPING region server 'jenkins-hbase4.apache.org,42073,1702191353745' ***** 2023-12-10 07:03:23,996 INFO [Listener at localhost/38369] regionserver.HRegionServer(2314): STOPPED: Shutdown requested 2023-12-10 07:03:23,996 INFO [RS:0;jenkins-hbase4:37997] regionserver.HeapMemoryManager(220): Stopping 2023-12-10 07:03:23,996 INFO [RS:1;jenkins-hbase4:42073] regionserver.HeapMemoryManager(220): Stopping 2023-12-10 07:03:23,996 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-12-10 07:03:23,996 INFO [RS:0;jenkins-hbase4:37997] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-12-10 07:03:23,996 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-12-10 07:03:23,996 INFO [RS:1;jenkins-hbase4:42073] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-12-10 07:03:23,996 INFO [RS:0;jenkins-hbase4:37997] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-12-10 07:03:23,997 INFO [RS:1;jenkins-hbase4:42073] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-12-10 07:03:23,997 INFO [RS:1;jenkins-hbase4:42073] regionserver.HRegionServer(3308): Received CLOSE for dca2e6ecbeef49fb8929fd36dd74df8b 2023-12-10 07:03:23,997 INFO [RS:0;jenkins-hbase4:37997] regionserver.HRegionServer(3308): Received CLOSE for 3b6714111b5dc2b8d14208f4c54166e3 2023-12-10 07:03:23,997 INFO [RS:0;jenkins-hbase4:37997] regionserver.HRegionServer(1147): stopping server jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 07:03:23,997 INFO [RS:1;jenkins-hbase4:42073] regionserver.HRegionServer(1147): stopping server jenkins-hbase4.apache.org,42073,1702191353745 2023-12-10 07:03:23,998 INFO [RS:1;jenkins-hbase4:42073] client.ConnectionImplementation(1973): Closing master protocol: MasterService 2023-12-10 07:03:23,998 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 3b6714111b5dc2b8d14208f4c54166e3, disabling compactions & flushes 2023-12-10 07:03:23,998 INFO [RS:0;jenkins-hbase4:37997] client.ConnectionImplementation(1973): Closing master protocol: MasterService 2023-12-10 07:03:23,998 DEBUG [RS:1;jenkins-hbase4:42073] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7978c06b to 127.0.0.1:51670 2023-12-10 07:03:23,998 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1702191355536.3b6714111b5dc2b8d14208f4c54166e3. 2023-12-10 07:03:23,998 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing dca2e6ecbeef49fb8929fd36dd74df8b, disabling compactions & flushes 2023-12-10 07:03:23,998 DEBUG [RS:0;jenkins-hbase4:37997] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1e40ef1e to 127.0.0.1:51670 2023-12-10 07:03:23,998 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b. 2023-12-10 07:03:23,998 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1702191355536.3b6714111b5dc2b8d14208f4c54166e3. 2023-12-10 07:03:23,998 DEBUG [RS:1;jenkins-hbase4:42073] ipc.AbstractRpcClient(489): Stopping rpc client 2023-12-10 07:03:23,998 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1702191355536.3b6714111b5dc2b8d14208f4c54166e3. after waiting 0 ms 2023-12-10 07:03:23,998 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1702191355536.3b6714111b5dc2b8d14208f4c54166e3. 2023-12-10 07:03:23,998 INFO [RS:1;jenkins-hbase4:42073] regionserver.HRegionServer(1477): Waiting on 1 regions to close 2023-12-10 07:03:23,998 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b. 2023-12-10 07:03:23,998 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b. after waiting 0 ms 2023-12-10 07:03:23,998 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b. 2023-12-10 07:03:23,998 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing dca2e6ecbeef49fb8929fd36dd74df8b 2/2 column families, dataSize=60 B heapSize=720 B 2023-12-10 07:03:23,998 DEBUG [RS:0;jenkins-hbase4:37997] ipc.AbstractRpcClient(489): Stopping rpc client 2023-12-10 07:03:23,998 DEBUG [RS:1;jenkins-hbase4:42073] regionserver.HRegionServer(1481): Online Regions={dca2e6ecbeef49fb8929fd36dd74df8b=hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b.} 2023-12-10 07:03:23,998 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 3b6714111b5dc2b8d14208f4c54166e3 1/1 column families, dataSize=30 B heapSize=360 B 2023-12-10 07:03:23,999 INFO [RS:0;jenkins-hbase4:37997] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-12-10 07:03:23,999 INFO [RS:0;jenkins-hbase4:37997] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-12-10 07:03:23,999 INFO [RS:0;jenkins-hbase4:37997] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-12-10 07:03:23,999 INFO [RS:0;jenkins-hbase4:37997] regionserver.HRegionServer(3308): Received CLOSE for 1588230740 2023-12-10 07:03:23,999 INFO [RS:0;jenkins-hbase4:37997] regionserver.HRegionServer(1477): Waiting on 2 regions to close 2023-12-10 07:03:24,000 DEBUG [RS:0;jenkins-hbase4:37997] regionserver.HRegionServer(1481): Online Regions={1588230740=hbase:meta,,1.1588230740, 3b6714111b5dc2b8d14208f4c54166e3=hbase:namespace,,1702191355536.3b6714111b5dc2b8d14208f4c54166e3.} 2023-12-10 07:03:24,000 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-12-10 07:03:24,000 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-12-10 07:03:24,000 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-12-10 07:03:24,000 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-12-10 07:03:24,000 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-12-10 07:03:24,000 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=5.37 KB heapSize=10.30 KB 2023-12-10 07:03:24,001 DEBUG [RS:1;jenkins-hbase4:42073] regionserver.HRegionServer(1507): Waiting on dca2e6ecbeef49fb8929fd36dd74df8b 2023-12-10 07:03:24,001 DEBUG [RS:0;jenkins-hbase4:37997] regionserver.HRegionServer(1507): Waiting on 1588230740, 3b6714111b5dc2b8d14208f4c54166e3 2023-12-10 07:03:24,027 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=30 B at sequenceid=11 (bloomFilter=true), to=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/namespace/3b6714111b5dc2b8d14208f4c54166e3/.tmp/info/cf0103ecb7b2429c98d9445494bf870c 2023-12-10 07:03:24,036 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=30 B at sequenceid=38 (bloomFilter=true), to=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/.tmp/q/51665561d3794435a6efebf53d830c47 2023-12-10 07:03:24,038 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cf0103ecb7b2429c98d9445494bf870c 2023-12-10 07:03:24,039 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/namespace/3b6714111b5dc2b8d14208f4c54166e3/.tmp/info/cf0103ecb7b2429c98d9445494bf870c as hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/namespace/3b6714111b5dc2b8d14208f4c54166e3/info/cf0103ecb7b2429c98d9445494bf870c 2023-12-10 07:03:24,041 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.92 KB at sequenceid=84 (bloomFilter=false), to=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740/.tmp/info/00e2262568bf4de3ab4bbbbb74bac327 2023-12-10 07:03:24,043 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 51665561d3794435a6efebf53d830c47 2023-12-10 07:03:24,046 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cf0103ecb7b2429c98d9445494bf870c 2023-12-10 07:03:24,046 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/namespace/3b6714111b5dc2b8d14208f4c54166e3/info/cf0103ecb7b2429c98d9445494bf870c, entries=1, sequenceid=11, filesize=4.9 K 2023-12-10 07:03:24,048 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 00e2262568bf4de3ab4bbbbb74bac327 2023-12-10 07:03:24,048 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~30 B/30, heapSize ~344 B/344, currentSize=0 B/0 for 3b6714111b5dc2b8d14208f4c54166e3 in 50ms, sequenceid=11, compaction requested=false 2023-12-10 07:03:24,059 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/namespace/3b6714111b5dc2b8d14208f4c54166e3/recovered.edits/14.seqid, newMaxSeqId=14, maxSeqId=1 2023-12-10 07:03:24,060 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-12-10 07:03:24,060 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-12-10 07:03:24,061 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1702191355536.3b6714111b5dc2b8d14208f4c54166e3. 2023-12-10 07:03:24,061 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 3b6714111b5dc2b8d14208f4c54166e3: 2023-12-10 07:03:24,061 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=30 B at sequenceid=38 (bloomFilter=true), to=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/.tmp/u/abbb15c1b10d4552ab0874f8720ece2e 2023-12-10 07:03:24,062 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1702191355536.3b6714111b5dc2b8d14208f4c54166e3. 2023-12-10 07:03:24,067 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=478 B at sequenceid=84 (bloomFilter=false), to=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740/.tmp/rep_barrier/aba4adba8c4e4006a2c1368357489bfa 2023-12-10 07:03:24,069 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for abbb15c1b10d4552ab0874f8720ece2e 2023-12-10 07:03:24,070 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/.tmp/q/51665561d3794435a6efebf53d830c47 as hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/q/51665561d3794435a6efebf53d830c47 2023-12-10 07:03:24,072 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aba4adba8c4e4006a2c1368357489bfa 2023-12-10 07:03:24,077 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 51665561d3794435a6efebf53d830c47 2023-12-10 07:03:24,077 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/q/51665561d3794435a6efebf53d830c47, entries=1, sequenceid=38, filesize=4.9 K 2023-12-10 07:03:24,078 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/.tmp/u/abbb15c1b10d4552ab0874f8720ece2e as hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/u/abbb15c1b10d4552ab0874f8720ece2e 2023-12-10 07:03:24,085 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1007 B at sequenceid=84 (bloomFilter=false), to=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740/.tmp/table/6a5fb0a524bb44fa817d78c100579b32 2023-12-10 07:03:24,086 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for abbb15c1b10d4552ab0874f8720ece2e 2023-12-10 07:03:24,086 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/u/abbb15c1b10d4552ab0874f8720ece2e, entries=1, sequenceid=38, filesize=4.9 K 2023-12-10 07:03:24,087 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~60 B/60, heapSize ~688 B/688, currentSize=0 B/0 for dca2e6ecbeef49fb8929fd36dd74df8b in 89ms, sequenceid=38, compaction requested=false 2023-12-10 07:03:24,100 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6a5fb0a524bb44fa817d78c100579b32 2023-12-10 07:03:24,101 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/quota/dca2e6ecbeef49fb8929fd36dd74df8b/recovered.edits/41.seqid, newMaxSeqId=41, maxSeqId=17 2023-12-10 07:03:24,101 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740/.tmp/info/00e2262568bf4de3ab4bbbbb74bac327 as hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740/info/00e2262568bf4de3ab4bbbbb74bac327 2023-12-10 07:03:24,102 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b. 2023-12-10 07:03:24,102 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for dca2e6ecbeef49fb8929fd36dd74df8b: 2023-12-10 07:03:24,102 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:quota,,1702191356247.dca2e6ecbeef49fb8929fd36dd74df8b. 2023-12-10 07:03:24,109 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 00e2262568bf4de3ab4bbbbb74bac327 2023-12-10 07:03:24,109 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740/info/00e2262568bf4de3ab4bbbbb74bac327, entries=10, sequenceid=84, filesize=5.7 K 2023-12-10 07:03:24,110 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740/.tmp/rep_barrier/aba4adba8c4e4006a2c1368357489bfa as hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740/rep_barrier/aba4adba8c4e4006a2c1368357489bfa 2023-12-10 07:03:24,116 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aba4adba8c4e4006a2c1368357489bfa 2023-12-10 07:03:24,116 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740/rep_barrier/aba4adba8c4e4006a2c1368357489bfa, entries=5, sequenceid=84, filesize=5.3 K 2023-12-10 07:03:24,117 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740/.tmp/table/6a5fb0a524bb44fa817d78c100579b32 as hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740/table/6a5fb0a524bb44fa817d78c100579b32 2023-12-10 07:03:24,121 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6a5fb0a524bb44fa817d78c100579b32 2023-12-10 07:03:24,121 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740/table/6a5fb0a524bb44fa817d78c100579b32, entries=9, sequenceid=84, filesize=5.4 K 2023-12-10 07:03:24,123 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~5.37 KB/5502, heapSize ~10.25 KB/10496, currentSize=0 B/0 for 1588230740 in 123ms, sequenceid=84, compaction requested=false 2023-12-10 07:03:24,130 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/data/hbase/meta/1588230740/recovered.edits/87.seqid, newMaxSeqId=87, maxSeqId=1 2023-12-10 07:03:24,131 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-12-10 07:03:24,131 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-12-10 07:03:24,131 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-12-10 07:03:24,131 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-12-10 07:03:24,201 INFO [RS:1;jenkins-hbase4:42073] regionserver.HRegionServer(1173): stopping server jenkins-hbase4.apache.org,42073,1702191353745; all regions closed. 2023-12-10 07:03:24,201 INFO [RS:0;jenkins-hbase4:37997] regionserver.HRegionServer(1173): stopping server jenkins-hbase4.apache.org,37997,1702191353702; all regions closed. 2023-12-10 07:03:24,201 DEBUG [RS:0;jenkins-hbase4:37997] quotas.QuotaCache(100): Stopping QuotaRefresherChore chore. 2023-12-10 07:03:24,201 DEBUG [RS:1;jenkins-hbase4:42073] quotas.QuotaCache(100): Stopping QuotaRefresherChore chore. 2023-12-10 07:03:24,212 DEBUG [RS:0;jenkins-hbase4:37997] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/oldWALs 2023-12-10 07:03:24,212 INFO [RS:0;jenkins-hbase4:37997] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C37997%2C1702191353702.meta:.meta(num 1702191355293) 2023-12-10 07:03:24,212 DEBUG [RS:1;jenkins-hbase4:42073] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/oldWALs 2023-12-10 07:03:24,212 INFO [RS:1;jenkins-hbase4:42073] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C42073%2C1702191353745:(num 1702191355101) 2023-12-10 07:03:24,212 DEBUG [RS:1;jenkins-hbase4:42073] ipc.AbstractRpcClient(489): Stopping rpc client 2023-12-10 07:03:24,212 INFO [RS:1;jenkins-hbase4:42073] regionserver.LeaseManager(133): Closed leases 2023-12-10 07:03:24,213 INFO [RS:1;jenkins-hbase4:42073] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-12-10 07:03:24,213 INFO [RS:1;jenkins-hbase4:42073] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-12-10 07:03:24,213 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-12-10 07:03:24,214 INFO [RS:1;jenkins-hbase4:42073] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-12-10 07:03:24,214 INFO [RS:1;jenkins-hbase4:42073] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-12-10 07:03:24,216 INFO [RS:1;jenkins-hbase4:42073] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:42073 2023-12-10 07:03:24,219 DEBUG [RS:0;jenkins-hbase4:37997] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/oldWALs 2023-12-10 07:03:24,219 INFO [RS:0;jenkins-hbase4:37997] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C37997%2C1702191353702:(num 1702191355101) 2023-12-10 07:03:24,219 DEBUG [RS:0;jenkins-hbase4:37997] ipc.AbstractRpcClient(489): Stopping rpc client 2023-12-10 07:03:24,219 INFO [RS:0;jenkins-hbase4:37997] regionserver.LeaseManager(133): Closed leases 2023-12-10 07:03:24,219 INFO [RS:0;jenkins-hbase4:37997] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-12-10 07:03:24,220 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-12-10 07:03:24,220 INFO [RS:0;jenkins-hbase4:37997] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:37997 2023-12-10 07:03:24,227 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-12-10 07:03:24,227 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): regionserver:42073-0x1007092abbd0002, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 07:03:24,227 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): regionserver:42073-0x1007092abbd0002, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-12-10 07:03:24,228 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): regionserver:42073-0x1007092abbd0002, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,42073,1702191353745 2023-12-10 07:03:24,228 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): regionserver:37997-0x1007092abbd0001, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,37997,1702191353702 2023-12-10 07:03:24,228 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): regionserver:37997-0x1007092abbd0001, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-12-10 07:03:24,228 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): regionserver:37997-0x1007092abbd0001, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,42073,1702191353745 2023-12-10 07:03:24,228 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,37997,1702191353702] 2023-12-10 07:03:24,229 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,37997,1702191353702; numProcessing=1 2023-12-10 07:03:24,231 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,37997,1702191353702 already deleted, retry=false 2023-12-10 07:03:24,231 INFO [RegionServerTracker-0] master.ServerManager(554): Cluster shutdown set; jenkins-hbase4.apache.org,37997,1702191353702 expired; onlineServers=1 2023-12-10 07:03:24,231 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,42073,1702191353745] 2023-12-10 07:03:24,231 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,42073,1702191353745; numProcessing=2 2023-12-10 07:03:24,233 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,42073,1702191353745 already deleted, retry=false 2023-12-10 07:03:24,233 INFO [RegionServerTracker-0] master.ServerManager(554): Cluster shutdown set; jenkins-hbase4.apache.org,42073,1702191353745 expired; onlineServers=0 2023-12-10 07:03:24,233 INFO [RegionServerTracker-0] regionserver.HRegionServer(2300): ***** STOPPING region server 'jenkins-hbase4.apache.org,40533,1702191352570' ***** 2023-12-10 07:03:24,233 INFO [RegionServerTracker-0] regionserver.HRegionServer(2314): STOPPED: Cluster shutdown set; onlineServer=0 2023-12-10 07:03:24,233 DEBUG [M:0;jenkins-hbase4:40533] ipc.AbstractRpcClient(189): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@27fd869d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-12-10 07:03:24,233 INFO [M:0;jenkins-hbase4:40533] regionserver.HRegionServer(1147): stopping server jenkins-hbase4.apache.org,40533,1702191352570 2023-12-10 07:03:24,233 INFO [M:0;jenkins-hbase4:40533] regionserver.HRegionServer(1173): stopping server jenkins-hbase4.apache.org,40533,1702191352570; all regions closed. 2023-12-10 07:03:24,233 DEBUG [M:0;jenkins-hbase4:40533] ipc.AbstractRpcClient(489): Stopping rpc client 2023-12-10 07:03:24,233 DEBUG [M:0;jenkins-hbase4:40533] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-12-10 07:03:24,234 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-12-10 07:03:24,234 DEBUG [M:0;jenkins-hbase4:40533] cleaner.HFileCleaner(317): Stopping file delete threads 2023-12-10 07:03:24,234 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1702191354790] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1702191354790,5,FailOnTimeoutGroup] 2023-12-10 07:03:24,234 INFO [M:0;jenkins-hbase4:40533] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-12-10 07:03:24,234 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1702191354790] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1702191354790,5,FailOnTimeoutGroup] 2023-12-10 07:03:24,234 INFO [M:0;jenkins-hbase4:40533] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-12-10 07:03:24,234 INFO [M:0;jenkins-hbase4:40533] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [ScheduledChore name=QuotaObserverChore, period=60000, unit=MILLISECONDS] on shutdown 2023-12-10 07:03:24,234 DEBUG [M:0;jenkins-hbase4:40533] master.HMaster(1517): Stopping service threads 2023-12-10 07:03:24,234 INFO [M:0;jenkins-hbase4:40533] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-12-10 07:03:24,235 ERROR [M:0;jenkins-hbase4:40533] procedure2.ProcedureExecutor(699): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[IPC Client (35059258) connection to localhost/127.0.0.1:39271 from jenkins,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-12-10 07:03:24,235 INFO [M:0;jenkins-hbase4:40533] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-12-10 07:03:24,235 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-12-10 07:03:24,236 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-12-10 07:03:24,236 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-12-10 07:03:24,236 DEBUG [M:0;jenkins-hbase4:40533] zookeeper.ZKUtil(399): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-12-10 07:03:24,236 WARN [M:0;jenkins-hbase4:40533] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-12-10 07:03:24,236 INFO [M:0;jenkins-hbase4:40533] assignment.AssignmentManager(315): Stopping assignment manager 2023-12-10 07:03:24,237 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(165): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-12-10 07:03:24,237 INFO [M:0;jenkins-hbase4:40533] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-12-10 07:03:24,237 DEBUG [M:0;jenkins-hbase4:40533] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-12-10 07:03:24,237 INFO [M:0;jenkins-hbase4:40533] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-12-10 07:03:24,237 DEBUG [M:0;jenkins-hbase4:40533] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-12-10 07:03:24,237 DEBUG [M:0;jenkins-hbase4:40533] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-12-10 07:03:24,237 DEBUG [M:0;jenkins-hbase4:40533] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-12-10 07:03:24,237 INFO [M:0;jenkins-hbase4:40533] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=183.44 KB heapSize=223.31 KB 2023-12-10 07:03:24,250 INFO [M:0;jenkins-hbase4:40533] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=183.44 KB at sequenceid=464 (bloomFilter=true), to=hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/2e1ce9a7cc9241e38cf4b7e0592b4fe6 2023-12-10 07:03:24,255 INFO [M:0;jenkins-hbase4:40533] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2e1ce9a7cc9241e38cf4b7e0592b4fe6 2023-12-10 07:03:24,256 DEBUG [M:0;jenkins-hbase4:40533] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/2e1ce9a7cc9241e38cf4b7e0592b4fe6 as hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/2e1ce9a7cc9241e38cf4b7e0592b4fe6 2023-12-10 07:03:24,260 INFO [M:0;jenkins-hbase4:40533] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2e1ce9a7cc9241e38cf4b7e0592b4fe6 2023-12-10 07:03:24,260 INFO [M:0;jenkins-hbase4:40533] regionserver.HStore(1080): Added hdfs://localhost:39271/user/jenkins/test-data/51f619ac-ce85-4f15-fc12-98d2f2826cb2/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/2e1ce9a7cc9241e38cf4b7e0592b4fe6, entries=54, sequenceid=464, filesize=9.5 K 2023-12-10 07:03:24,261 INFO [M:0;jenkins-hbase4:40533] regionserver.HRegion(2948): Finished flush of dataSize ~183.44 KB/187841, heapSize ~223.30 KB/228656, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 24ms, sequenceid=464, compaction requested=false 2023-12-10 07:03:24,262 INFO [M:0;jenkins-hbase4:40533] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-12-10 07:03:24,262 DEBUG [M:0;jenkins-hbase4:40533] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-12-10 07:03:24,265 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-12-10 07:03:24,265 INFO [M:0;jenkins-hbase4:40533] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-12-10 07:03:24,266 INFO [M:0;jenkins-hbase4:40533] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:40533 2023-12-10 07:03:24,268 DEBUG [M:0;jenkins-hbase4:40533] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,40533,1702191352570 already deleted, retry=false 2023-12-10 07:03:24,331 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): regionserver:37997-0x1007092abbd0001, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-12-10 07:03:24,331 INFO [RS:0;jenkins-hbase4:37997] regionserver.HRegionServer(1230): Exiting; stopping=jenkins-hbase4.apache.org,37997,1702191353702; zookeeper connection closed. 2023-12-10 07:03:24,331 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): regionserver:42073-0x1007092abbd0002, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-12-10 07:03:24,331 INFO [RS:1;jenkins-hbase4:42073] regionserver.HRegionServer(1230): Exiting; stopping=jenkins-hbase4.apache.org,42073,1702191353745; zookeeper connection closed. 2023-12-10 07:03:24,331 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): regionserver:42073-0x1007092abbd0002, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-12-10 07:03:24,331 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): regionserver:37997-0x1007092abbd0001, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-12-10 07:03:24,331 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@1ca4391b] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@1ca4391b 2023-12-10 07:03:24,331 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@8749add] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@8749add 2023-12-10 07:03:24,331 INFO [Listener at localhost/38369] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 2 regionserver(s) complete 2023-12-10 07:03:24,369 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-12-10 07:03:24,369 INFO [M:0;jenkins-hbase4:40533] regionserver.HRegionServer(1230): Exiting; stopping=jenkins-hbase4.apache.org,40533,1702191352570; zookeeper connection closed. 2023-12-10 07:03:24,369 DEBUG [Listener at localhost/38369-EventThread] zookeeper.ZKWatcher(600): master:40533-0x1007092abbd0000, quorum=127.0.0.1:51670, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-12-10 07:03:24,371 WARN [Listener at localhost/38369] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-12-10 07:03:24,373 INFO [Listener at localhost/38369] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-12-10 07:03:24,477 WARN [BP-1696142428-172.31.14.131-1702191349506 heartbeating to localhost/127.0.0.1:39271] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-12-10 07:03:24,477 WARN [BP-1696142428-172.31.14.131-1702191349506 heartbeating to localhost/127.0.0.1:39271] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1696142428-172.31.14.131-1702191349506 (Datanode Uuid f6ef735b-668b-4f1c-b700-62801aa08b11) service to localhost/127.0.0.1:39271 2023-12-10 07:03:24,479 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/32556c7f-b21a-6d60-9914-e7d129008d6d/cluster_4172d4ef-86cd-605e-0704-d776bf6f94c3/dfs/data/data3/current/BP-1696142428-172.31.14.131-1702191349506] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-12-10 07:03:24,479 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/32556c7f-b21a-6d60-9914-e7d129008d6d/cluster_4172d4ef-86cd-605e-0704-d776bf6f94c3/dfs/data/data4/current/BP-1696142428-172.31.14.131-1702191349506] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-12-10 07:03:24,480 WARN [Listener at localhost/38369] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-12-10 07:03:24,482 INFO [Listener at localhost/38369] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-12-10 07:03:24,586 WARN [BP-1696142428-172.31.14.131-1702191349506 heartbeating to localhost/127.0.0.1:39271] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-12-10 07:03:24,586 WARN [BP-1696142428-172.31.14.131-1702191349506 heartbeating to localhost/127.0.0.1:39271] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1696142428-172.31.14.131-1702191349506 (Datanode Uuid 2ffdebb3-2bf6-4e4c-8e40-1e05648d3392) service to localhost/127.0.0.1:39271 2023-12-10 07:03:24,586 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/32556c7f-b21a-6d60-9914-e7d129008d6d/cluster_4172d4ef-86cd-605e-0704-d776bf6f94c3/dfs/data/data1/current/BP-1696142428-172.31.14.131-1702191349506] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-12-10 07:03:24,587 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/32556c7f-b21a-6d60-9914-e7d129008d6d/cluster_4172d4ef-86cd-605e-0704-d776bf6f94c3/dfs/data/data2/current/BP-1696142428-172.31.14.131-1702191349506] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-12-10 07:03:24,618 INFO [Listener at localhost/38369] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-12-10 07:03:24,733 INFO [Listener at localhost/38369] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-12-10 07:03:24,775 INFO [Listener at localhost/38369] hbase.HBaseTestingUtility(1293): Minicluster is down