2023-07-24 20:10:32,206 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe 2023-07-24 20:10:32,223 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics timeout: 13 mins 2023-07-24 20:10:32,242 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-07-24 20:10:32,243 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/cluster_82e812c0-1679-84b5-414c-93cfc125cc3f, deleteOnExit=true 2023-07-24 20:10:32,243 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-07-24 20:10:32,244 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/test.cache.data in system properties and HBase conf 2023-07-24 20:10:32,244 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/hadoop.tmp.dir in system properties and HBase conf 2023-07-24 20:10:32,245 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/hadoop.log.dir in system properties and HBase conf 2023-07-24 20:10:32,245 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/mapreduce.cluster.local.dir in system properties and HBase conf 2023-07-24 20:10:32,245 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-07-24 20:10:32,246 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-07-24 20:10:32,390 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-07-24 20:10:32,881 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-07-24 20:10:32,887 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-07-24 20:10:32,888 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-07-24 20:10:32,888 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-07-24 20:10:32,888 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-24 20:10:32,888 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-07-24 20:10:32,889 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-07-24 20:10:32,889 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-24 20:10:32,890 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-24 20:10:32,890 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-07-24 20:10:32,890 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/nfs.dump.dir in system properties and HBase conf 2023-07-24 20:10:32,890 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/java.io.tmpdir in system properties and HBase conf 2023-07-24 20:10:32,891 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-24 20:10:32,891 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-07-24 20:10:32,891 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-07-24 20:10:33,522 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-24 20:10:33,527 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-24 20:10:33,898 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-07-24 20:10:34,128 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-07-24 20:10:34,144 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-24 20:10:34,186 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-07-24 20:10:34,221 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/java.io.tmpdir/Jetty_localhost_37109_hdfs____.lax0lc/webapp 2023-07-24 20:10:34,386 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:37109 2023-07-24 20:10:34,397 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-24 20:10:34,397 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-24 20:10:34,938 WARN [Listener at localhost/37609] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-24 20:10:35,034 WARN [Listener at localhost/37609] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-24 20:10:35,107 WARN [Listener at localhost/37609] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-24 20:10:35,116 INFO [Listener at localhost/37609] log.Slf4jLog(67): jetty-6.1.26 2023-07-24 20:10:35,123 INFO [Listener at localhost/37609] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/java.io.tmpdir/Jetty_localhost_44589_datanode____.6j5ler/webapp 2023-07-24 20:10:35,232 INFO [Listener at localhost/37609] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44589 2023-07-24 20:10:35,650 WARN [Listener at localhost/42623] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-24 20:10:35,688 WARN [Listener at localhost/42623] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-24 20:10:35,699 WARN [Listener at localhost/42623] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-24 20:10:35,701 INFO [Listener at localhost/42623] log.Slf4jLog(67): jetty-6.1.26 2023-07-24 20:10:35,711 INFO [Listener at localhost/42623] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/java.io.tmpdir/Jetty_localhost_36715_datanode____ayx16n/webapp 2023-07-24 20:10:35,839 INFO [Listener at localhost/42623] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:36715 2023-07-24 20:10:35,858 WARN [Listener at localhost/34163] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-24 20:10:35,890 WARN [Listener at localhost/34163] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-24 20:10:35,894 WARN [Listener at localhost/34163] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-24 20:10:35,896 INFO [Listener at localhost/34163] log.Slf4jLog(67): jetty-6.1.26 2023-07-24 20:10:35,905 INFO [Listener at localhost/34163] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/java.io.tmpdir/Jetty_localhost_35373_datanode____la9fdg/webapp 2023-07-24 20:10:36,036 INFO [Listener at localhost/34163] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35373 2023-07-24 20:10:36,047 WARN [Listener at localhost/44145] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-24 20:10:36,314 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x77f0e765f26c5bdf: Processing first storage report for DS-d723b047-8a44-4a99-b3c7-f9704403895f from datanode 3e40cac9-ace0-417b-a97f-3a6f56ab9729 2023-07-24 20:10:36,315 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x77f0e765f26c5bdf: from storage DS-d723b047-8a44-4a99-b3c7-f9704403895f node DatanodeRegistration(127.0.0.1:34425, datanodeUuid=3e40cac9-ace0-417b-a97f-3a6f56ab9729, infoPort=32867, infoSecurePort=0, ipcPort=34163, storageInfo=lv=-57;cid=testClusterID;nsid=38439225;c=1690229433619), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-07-24 20:10:36,316 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xbacd63f263b08ef5: Processing first storage report for DS-a5b4792c-4be5-4b56-947a-d97b275970c9 from datanode e3e2f909-b3c6-4182-9f15-560a4d523f97 2023-07-24 20:10:36,316 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xbacd63f263b08ef5: from storage DS-a5b4792c-4be5-4b56-947a-d97b275970c9 node DatanodeRegistration(127.0.0.1:37501, datanodeUuid=e3e2f909-b3c6-4182-9f15-560a4d523f97, infoPort=43525, infoSecurePort=0, ipcPort=44145, storageInfo=lv=-57;cid=testClusterID;nsid=38439225;c=1690229433619), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-24 20:10:36,316 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xa9da5ac44f87b5bb: Processing first storage report for DS-062e8295-abc1-45d4-88c5-80d70c08380a from datanode 32e0b698-571a-4a8e-8396-ca5cd6d8f1ca 2023-07-24 20:10:36,316 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xa9da5ac44f87b5bb: from storage DS-062e8295-abc1-45d4-88c5-80d70c08380a node DatanodeRegistration(127.0.0.1:38567, datanodeUuid=32e0b698-571a-4a8e-8396-ca5cd6d8f1ca, infoPort=38749, infoSecurePort=0, ipcPort=42623, storageInfo=lv=-57;cid=testClusterID;nsid=38439225;c=1690229433619), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-24 20:10:36,316 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x77f0e765f26c5bdf: Processing first storage report for DS-374e2107-0240-4dd3-895a-6539f8186182 from datanode 3e40cac9-ace0-417b-a97f-3a6f56ab9729 2023-07-24 20:10:36,316 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x77f0e765f26c5bdf: from storage DS-374e2107-0240-4dd3-895a-6539f8186182 node DatanodeRegistration(127.0.0.1:34425, datanodeUuid=3e40cac9-ace0-417b-a97f-3a6f56ab9729, infoPort=32867, infoSecurePort=0, ipcPort=34163, storageInfo=lv=-57;cid=testClusterID;nsid=38439225;c=1690229433619), blocks: 0, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 0 2023-07-24 20:10:36,317 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xbacd63f263b08ef5: Processing first storage report for DS-4600919c-5ba3-4473-9a32-f5a01c945d88 from datanode e3e2f909-b3c6-4182-9f15-560a4d523f97 2023-07-24 20:10:36,317 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xbacd63f263b08ef5: from storage DS-4600919c-5ba3-4473-9a32-f5a01c945d88 node DatanodeRegistration(127.0.0.1:37501, datanodeUuid=e3e2f909-b3c6-4182-9f15-560a4d523f97, infoPort=43525, infoSecurePort=0, ipcPort=44145, storageInfo=lv=-57;cid=testClusterID;nsid=38439225;c=1690229433619), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-24 20:10:36,317 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xa9da5ac44f87b5bb: Processing first storage report for DS-8b05f92d-849b-4703-82e7-7106fc609095 from datanode 32e0b698-571a-4a8e-8396-ca5cd6d8f1ca 2023-07-24 20:10:36,317 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xa9da5ac44f87b5bb: from storage DS-8b05f92d-849b-4703-82e7-7106fc609095 node DatanodeRegistration(127.0.0.1:38567, datanodeUuid=32e0b698-571a-4a8e-8396-ca5cd6d8f1ca, infoPort=38749, infoSecurePort=0, ipcPort=42623, storageInfo=lv=-57;cid=testClusterID;nsid=38439225;c=1690229433619), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-24 20:10:36,554 DEBUG [Listener at localhost/44145] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe 2023-07-24 20:10:36,639 INFO [Listener at localhost/44145] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/cluster_82e812c0-1679-84b5-414c-93cfc125cc3f/zookeeper_0, clientPort=62881, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/cluster_82e812c0-1679-84b5-414c-93cfc125cc3f/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/cluster_82e812c0-1679-84b5-414c-93cfc125cc3f/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-07-24 20:10:36,657 INFO [Listener at localhost/44145] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=62881 2023-07-24 20:10:36,668 INFO [Listener at localhost/44145] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:10:36,670 INFO [Listener at localhost/44145] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:10:37,430 INFO [Listener at localhost/44145] util.FSUtils(471): Created version file at hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4 with version=8 2023-07-24 20:10:37,431 INFO [Listener at localhost/44145] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/hbase-staging 2023-07-24 20:10:37,442 DEBUG [Listener at localhost/44145] hbase.LocalHBaseCluster(134): Setting Master Port to random. 2023-07-24 20:10:37,442 DEBUG [Listener at localhost/44145] hbase.LocalHBaseCluster(141): Setting RegionServer Port to random. 2023-07-24 20:10:37,443 DEBUG [Listener at localhost/44145] hbase.LocalHBaseCluster(151): Setting RS InfoServer Port to random. 2023-07-24 20:10:37,443 DEBUG [Listener at localhost/44145] hbase.LocalHBaseCluster(159): Setting Master InfoServer Port to random. 2023-07-24 20:10:37,899 INFO [Listener at localhost/44145] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-07-24 20:10:38,538 INFO [Listener at localhost/44145] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-07-24 20:10:38,587 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 20:10:38,588 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-24 20:10:38,588 INFO [Listener at localhost/44145] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-24 20:10:38,588 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 20:10:38,588 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-24 20:10:38,736 INFO [Listener at localhost/44145] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-24 20:10:38,860 DEBUG [Listener at localhost/44145] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-07-24 20:10:38,985 INFO [Listener at localhost/44145] ipc.NettyRpcServer(120): Bind to /172.31.14.131:39247 2023-07-24 20:10:39,001 INFO [Listener at localhost/44145] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:10:39,003 INFO [Listener at localhost/44145] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:10:39,038 INFO [Listener at localhost/44145] zookeeper.RecoverableZooKeeper(93): Process identifier=master:39247 connecting to ZooKeeper ensemble=127.0.0.1:62881 2023-07-24 20:10:39,101 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:392470x0, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-24 20:10:39,104 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:39247-0x10198f4f3ce0000 connected 2023-07-24 20:10:39,132 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-24 20:10:39,133 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 20:10:39,136 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-24 20:10:39,149 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=39247 2023-07-24 20:10:39,149 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=39247 2023-07-24 20:10:39,150 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=39247 2023-07-24 20:10:39,150 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=39247 2023-07-24 20:10:39,151 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=39247 2023-07-24 20:10:39,191 INFO [Listener at localhost/44145] log.Log(170): Logging initialized @7758ms to org.apache.hbase.thirdparty.org.eclipse.jetty.util.log.Slf4jLog 2023-07-24 20:10:39,396 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2023-07-24 20:10:39,396 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter) 2023-07-24 20:10:39,397 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'securityheaders' (class=org.apache.hadoop.hbase.http.SecurityHeadersFilter) 2023-07-24 20:10:39,400 INFO [Listener at localhost/44145] http.HttpServer(879): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context master 2023-07-24 20:10:39,400 INFO [Listener at localhost/44145] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2023-07-24 20:10:39,400 INFO [Listener at localhost/44145] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2023-07-24 20:10:39,405 INFO [Listener at localhost/44145] http.HttpServer(783): ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint. 2023-07-24 20:10:39,478 INFO [Listener at localhost/44145] http.HttpServer(1146): Jetty bound to port 36023 2023-07-24 20:10:39,480 INFO [Listener at localhost/44145] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-24 20:10:39,515 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:10:39,519 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@45ca76be{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/hadoop.log.dir/,AVAILABLE} 2023-07-24 20:10:39,519 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:10:39,520 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@562b38c7{static,/static,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/static/,AVAILABLE} 2023-07-24 20:10:39,602 INFO [Listener at localhost/44145] webapp.StandardDescriptorProcessor(277): NO JSP Support for /, did not find org.apache.hbase.thirdparty.org.eclipse.jetty.jsp.JettyJspServlet 2023-07-24 20:10:39,622 INFO [Listener at localhost/44145] session.DefaultSessionIdManager(334): DefaultSessionIdManager workerName=node0 2023-07-24 20:10:39,623 INFO [Listener at localhost/44145] session.DefaultSessionIdManager(339): No SessionScavenger set, using defaults 2023-07-24 20:10:39,626 INFO [Listener at localhost/44145] session.HouseKeeper(132): node0 Scavenging every 660000ms 2023-07-24 20:10:39,641 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:10:39,695 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.w.WebAppContext@12930a{master,/,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/master/,AVAILABLE}{file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/master} 2023-07-24 20:10:39,721 INFO [Listener at localhost/44145] server.AbstractConnector(333): Started ServerConnector@309e9f1{HTTP/1.1, (http/1.1)}{0.0.0.0:36023} 2023-07-24 20:10:39,721 INFO [Listener at localhost/44145] server.Server(415): Started @8288ms 2023-07-24 20:10:39,734 INFO [Listener at localhost/44145] master.HMaster(444): hbase.rootdir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4, hbase.cluster.distributed=false 2023-07-24 20:10:39,859 INFO [Listener at localhost/44145] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-24 20:10:39,859 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 20:10:39,860 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-24 20:10:39,860 INFO [Listener at localhost/44145] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-24 20:10:39,861 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 20:10:39,861 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-24 20:10:39,873 INFO [Listener at localhost/44145] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-24 20:10:39,887 INFO [Listener at localhost/44145] ipc.NettyRpcServer(120): Bind to /172.31.14.131:34483 2023-07-24 20:10:39,891 INFO [Listener at localhost/44145] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-24 20:10:39,901 DEBUG [Listener at localhost/44145] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-24 20:10:39,902 INFO [Listener at localhost/44145] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:10:39,911 INFO [Listener at localhost/44145] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:10:39,913 INFO [Listener at localhost/44145] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:34483 connecting to ZooKeeper ensemble=127.0.0.1:62881 2023-07-24 20:10:39,919 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:344830x0, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-24 20:10:39,920 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): regionserver:344830x0, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-24 20:10:39,923 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:34483-0x10198f4f3ce0001 connected 2023-07-24 20:10:39,924 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 20:10:39,924 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-24 20:10:39,931 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34483 2023-07-24 20:10:39,931 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34483 2023-07-24 20:10:39,939 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34483 2023-07-24 20:10:39,958 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34483 2023-07-24 20:10:39,962 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34483 2023-07-24 20:10:39,966 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2023-07-24 20:10:39,966 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter) 2023-07-24 20:10:39,966 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'securityheaders' (class=org.apache.hadoop.hbase.http.SecurityHeadersFilter) 2023-07-24 20:10:39,967 INFO [Listener at localhost/44145] http.HttpServer(879): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context regionserver 2023-07-24 20:10:39,967 INFO [Listener at localhost/44145] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2023-07-24 20:10:39,967 INFO [Listener at localhost/44145] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2023-07-24 20:10:39,967 INFO [Listener at localhost/44145] http.HttpServer(783): ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint. 2023-07-24 20:10:39,970 INFO [Listener at localhost/44145] http.HttpServer(1146): Jetty bound to port 42205 2023-07-24 20:10:39,970 INFO [Listener at localhost/44145] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-24 20:10:39,976 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:10:39,977 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@15af0305{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/hadoop.log.dir/,AVAILABLE} 2023-07-24 20:10:39,977 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:10:39,978 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@5c57f5de{static,/static,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/static/,AVAILABLE} 2023-07-24 20:10:39,995 INFO [Listener at localhost/44145] webapp.StandardDescriptorProcessor(277): NO JSP Support for /, did not find org.apache.hbase.thirdparty.org.eclipse.jetty.jsp.JettyJspServlet 2023-07-24 20:10:39,996 INFO [Listener at localhost/44145] session.DefaultSessionIdManager(334): DefaultSessionIdManager workerName=node0 2023-07-24 20:10:39,997 INFO [Listener at localhost/44145] session.DefaultSessionIdManager(339): No SessionScavenger set, using defaults 2023-07-24 20:10:39,997 INFO [Listener at localhost/44145] session.HouseKeeper(132): node0 Scavenging every 600000ms 2023-07-24 20:10:39,999 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:10:40,005 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.w.WebAppContext@50a284fe{regionserver,/,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver/,AVAILABLE}{file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver} 2023-07-24 20:10:40,008 INFO [Listener at localhost/44145] server.AbstractConnector(333): Started ServerConnector@5417460a{HTTP/1.1, (http/1.1)}{0.0.0.0:42205} 2023-07-24 20:10:40,008 INFO [Listener at localhost/44145] server.Server(415): Started @8575ms 2023-07-24 20:10:40,022 INFO [Listener at localhost/44145] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-24 20:10:40,022 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 20:10:40,022 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-24 20:10:40,023 INFO [Listener at localhost/44145] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-24 20:10:40,023 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 20:10:40,023 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-24 20:10:40,023 INFO [Listener at localhost/44145] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-24 20:10:40,028 INFO [Listener at localhost/44145] ipc.NettyRpcServer(120): Bind to /172.31.14.131:34371 2023-07-24 20:10:40,028 INFO [Listener at localhost/44145] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-24 20:10:40,030 DEBUG [Listener at localhost/44145] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-24 20:10:40,031 INFO [Listener at localhost/44145] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:10:40,033 INFO [Listener at localhost/44145] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:10:40,035 INFO [Listener at localhost/44145] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:34371 connecting to ZooKeeper ensemble=127.0.0.1:62881 2023-07-24 20:10:40,044 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:343710x0, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-24 20:10:40,045 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:34371-0x10198f4f3ce0002 connected 2023-07-24 20:10:40,045 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): regionserver:34371-0x10198f4f3ce0002, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-24 20:10:40,046 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): regionserver:34371-0x10198f4f3ce0002, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 20:10:40,047 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): regionserver:34371-0x10198f4f3ce0002, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-24 20:10:40,049 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34371 2023-07-24 20:10:40,049 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34371 2023-07-24 20:10:40,050 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34371 2023-07-24 20:10:40,050 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34371 2023-07-24 20:10:40,050 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34371 2023-07-24 20:10:40,053 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2023-07-24 20:10:40,053 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter) 2023-07-24 20:10:40,053 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'securityheaders' (class=org.apache.hadoop.hbase.http.SecurityHeadersFilter) 2023-07-24 20:10:40,054 INFO [Listener at localhost/44145] http.HttpServer(879): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context regionserver 2023-07-24 20:10:40,054 INFO [Listener at localhost/44145] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2023-07-24 20:10:40,054 INFO [Listener at localhost/44145] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2023-07-24 20:10:40,054 INFO [Listener at localhost/44145] http.HttpServer(783): ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint. 2023-07-24 20:10:40,055 INFO [Listener at localhost/44145] http.HttpServer(1146): Jetty bound to port 34123 2023-07-24 20:10:40,055 INFO [Listener at localhost/44145] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-24 20:10:40,056 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:10:40,057 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@4cbd39c9{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/hadoop.log.dir/,AVAILABLE} 2023-07-24 20:10:40,057 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:10:40,057 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@64c0781{static,/static,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/static/,AVAILABLE} 2023-07-24 20:10:40,069 INFO [Listener at localhost/44145] webapp.StandardDescriptorProcessor(277): NO JSP Support for /, did not find org.apache.hbase.thirdparty.org.eclipse.jetty.jsp.JettyJspServlet 2023-07-24 20:10:40,070 INFO [Listener at localhost/44145] session.DefaultSessionIdManager(334): DefaultSessionIdManager workerName=node0 2023-07-24 20:10:40,070 INFO [Listener at localhost/44145] session.DefaultSessionIdManager(339): No SessionScavenger set, using defaults 2023-07-24 20:10:40,070 INFO [Listener at localhost/44145] session.HouseKeeper(132): node0 Scavenging every 660000ms 2023-07-24 20:10:40,071 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:10:40,072 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.w.WebAppContext@1fca8265{regionserver,/,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver/,AVAILABLE}{file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver} 2023-07-24 20:10:40,073 INFO [Listener at localhost/44145] server.AbstractConnector(333): Started ServerConnector@14c5b0ab{HTTP/1.1, (http/1.1)}{0.0.0.0:34123} 2023-07-24 20:10:40,073 INFO [Listener at localhost/44145] server.Server(415): Started @8640ms 2023-07-24 20:10:40,085 INFO [Listener at localhost/44145] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-24 20:10:40,085 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 20:10:40,085 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-24 20:10:40,085 INFO [Listener at localhost/44145] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-24 20:10:40,085 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 20:10:40,086 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-24 20:10:40,086 INFO [Listener at localhost/44145] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-24 20:10:40,088 INFO [Listener at localhost/44145] ipc.NettyRpcServer(120): Bind to /172.31.14.131:39791 2023-07-24 20:10:40,088 INFO [Listener at localhost/44145] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-24 20:10:40,090 DEBUG [Listener at localhost/44145] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-24 20:10:40,091 INFO [Listener at localhost/44145] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:10:40,093 INFO [Listener at localhost/44145] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:10:40,094 INFO [Listener at localhost/44145] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:39791 connecting to ZooKeeper ensemble=127.0.0.1:62881 2023-07-24 20:10:40,105 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:397910x0, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-24 20:10:40,106 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): regionserver:397910x0, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-24 20:10:40,107 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): regionserver:397910x0, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 20:10:40,108 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): regionserver:397910x0, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-24 20:10:40,111 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:39791-0x10198f4f3ce0003 connected 2023-07-24 20:10:40,114 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=39791 2023-07-24 20:10:40,114 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=39791 2023-07-24 20:10:40,117 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=39791 2023-07-24 20:10:40,118 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=39791 2023-07-24 20:10:40,118 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=39791 2023-07-24 20:10:40,122 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2023-07-24 20:10:40,122 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter) 2023-07-24 20:10:40,123 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'securityheaders' (class=org.apache.hadoop.hbase.http.SecurityHeadersFilter) 2023-07-24 20:10:40,123 INFO [Listener at localhost/44145] http.HttpServer(879): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context regionserver 2023-07-24 20:10:40,124 INFO [Listener at localhost/44145] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2023-07-24 20:10:40,124 INFO [Listener at localhost/44145] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2023-07-24 20:10:40,124 INFO [Listener at localhost/44145] http.HttpServer(783): ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint. 2023-07-24 20:10:40,125 INFO [Listener at localhost/44145] http.HttpServer(1146): Jetty bound to port 36539 2023-07-24 20:10:40,126 INFO [Listener at localhost/44145] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-24 20:10:40,128 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:10:40,128 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@57741f78{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/hadoop.log.dir/,AVAILABLE} 2023-07-24 20:10:40,129 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:10:40,129 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@3062dffd{static,/static,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/static/,AVAILABLE} 2023-07-24 20:10:40,138 INFO [Listener at localhost/44145] webapp.StandardDescriptorProcessor(277): NO JSP Support for /, did not find org.apache.hbase.thirdparty.org.eclipse.jetty.jsp.JettyJspServlet 2023-07-24 20:10:40,139 INFO [Listener at localhost/44145] session.DefaultSessionIdManager(334): DefaultSessionIdManager workerName=node0 2023-07-24 20:10:40,139 INFO [Listener at localhost/44145] session.DefaultSessionIdManager(339): No SessionScavenger set, using defaults 2023-07-24 20:10:40,140 INFO [Listener at localhost/44145] session.HouseKeeper(132): node0 Scavenging every 660000ms 2023-07-24 20:10:40,141 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:10:40,141 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.w.WebAppContext@1d722a93{regionserver,/,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver/,AVAILABLE}{file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver} 2023-07-24 20:10:40,143 INFO [Listener at localhost/44145] server.AbstractConnector(333): Started ServerConnector@7df64ca9{HTTP/1.1, (http/1.1)}{0.0.0.0:36539} 2023-07-24 20:10:40,143 INFO [Listener at localhost/44145] server.Server(415): Started @8710ms 2023-07-24 20:10:40,154 INFO [master/jenkins-hbase4:0:becomeActiveMaster] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-24 20:10:40,159 INFO [master/jenkins-hbase4:0:becomeActiveMaster] server.AbstractConnector(333): Started ServerConnector@4545c88a{HTTP/1.1, (http/1.1)}{0.0.0.0:42771} 2023-07-24 20:10:40,160 INFO [master/jenkins-hbase4:0:becomeActiveMaster] server.Server(415): Started @8727ms 2023-07-24 20:10:40,160 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,39247,1690229437650 2023-07-24 20:10:40,172 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-07-24 20:10:40,175 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,39247,1690229437650 2023-07-24 20:10:40,211 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-24 20:10:40,211 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34371-0x10198f4f3ce0002, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-24 20:10:40,212 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-24 20:10:40,213 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 20:10:40,212 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-24 20:10:40,217 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-24 20:10:40,220 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,39247,1690229437650 from backup master directory 2023-07-24 20:10:40,221 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-24 20:10:40,233 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,39247,1690229437650 2023-07-24 20:10:40,234 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-07-24 20:10:40,235 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-24 20:10:40,235 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,39247,1690229437650 2023-07-24 20:10:40,241 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-07-24 20:10:40,255 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-07-24 20:10:40,421 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/hbase.id with ID: dc54b028-3544-4133-8f0e-d550a19bc790 2023-07-24 20:10:40,473 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:10:40,492 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 20:10:40,557 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x4c458dfb to 127.0.0.1:62881 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-24 20:10:40,587 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@17c0b2c4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-24 20:10:40,616 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-24 20:10:40,618 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-24 20:10:40,641 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-07-24 20:10:40,642 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-07-24 20:10:40,644 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-07-24 20:10:40,650 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-07-24 20:10:40,652 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-24 20:10:40,694 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/data/master/store-tmp 2023-07-24 20:10:40,749 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:40,750 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-24 20:10:40,750 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-24 20:10:40,750 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-24 20:10:40,750 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-24 20:10:40,750 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-24 20:10:40,750 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-24 20:10:40,750 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-24 20:10:40,752 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/WALs/jenkins-hbase4.apache.org,39247,1690229437650 2023-07-24 20:10:40,783 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39247%2C1690229437650, suffix=, logDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/WALs/jenkins-hbase4.apache.org,39247,1690229437650, archiveDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/oldWALs, maxLogs=10 2023-07-24 20:10:40,849 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK] 2023-07-24 20:10:40,849 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK] 2023-07-24 20:10:40,849 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK] 2023-07-24 20:10:40,858 DEBUG [RS-EventLoopGroup-5-1] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-07-24 20:10:40,932 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/WALs/jenkins-hbase4.apache.org,39247,1690229437650/jenkins-hbase4.apache.org%2C39247%2C1690229437650.1690229440797 2023-07-24 20:10:40,932 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK], DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK], DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK]] 2023-07-24 20:10:40,933 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-24 20:10:40,933 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:40,937 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-24 20:10:40,938 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-24 20:10:41,026 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-24 20:10:41,035 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-24 20:10:41,076 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-24 20:10:41,094 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:10:41,100 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-24 20:10:41,102 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-24 20:10:41,130 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-24 20:10:41,145 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 20:10:41,147 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=11893868800, jitterRate=0.10770285129547119}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 20:10:41,147 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-24 20:10:41,155 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-24 20:10:41,186 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-24 20:10:41,186 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-24 20:10:41,189 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 20:10:41,192 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-07-24 20:10:41,239 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 46 msec 2023-07-24 20:10:41,239 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-24 20:10:41,263 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-07-24 20:10:41,270 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-07-24 20:10:41,278 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-07-24 20:10:41,285 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-24 20:10:41,293 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-07-24 20:10:41,296 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 20:10:41,297 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-07-24 20:10:41,298 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-07-24 20:10:41,313 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-07-24 20:10:41,319 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-24 20:10:41,319 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 20:10:41,319 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-24 20:10:41,319 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34371-0x10198f4f3ce0002, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-24 20:10:41,319 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-24 20:10:41,320 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,39247,1690229437650, sessionid=0x10198f4f3ce0000, setting cluster-up flag (Was=false) 2023-07-24 20:10:41,339 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 20:10:41,345 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-07-24 20:10:41,347 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,39247,1690229437650 2023-07-24 20:10:41,352 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 20:10:41,358 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-07-24 20:10:41,359 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,39247,1690229437650 2023-07-24 20:10:41,362 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.hbase-snapshot/.tmp 2023-07-24 20:10:41,439 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2930): Registered master coprocessor service: service=RSGroupAdminService 2023-07-24 20:10:41,449 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] rsgroup.RSGroupInfoManagerImpl(537): Refreshing in Offline mode. 2023-07-24 20:10:41,451 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,39247,1690229437650] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(787): Updating default servers. 2023-07-24 20:10:41,452 INFO [RS:0;jenkins-hbase4:34483] regionserver.HRegionServer(951): ClusterId : dc54b028-3544-4133-8f0e-d550a19bc790 2023-07-24 20:10:41,452 INFO [RS:1;jenkins-hbase4:34371] regionserver.HRegionServer(951): ClusterId : dc54b028-3544-4133-8f0e-d550a19bc790 2023-07-24 20:10:41,452 INFO [RS:2;jenkins-hbase4:39791] regionserver.HRegionServer(951): ClusterId : dc54b028-3544-4133-8f0e-d550a19bc790 2023-07-24 20:10:41,454 INFO [master/jenkins-hbase4:0:becomeActiveMaster] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint loaded, priority=536870911. 2023-07-24 20:10:41,455 INFO [master/jenkins-hbase4:0:becomeActiveMaster] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase$CPMasterObserver loaded, priority=536870912. 2023-07-24 20:10:41,459 DEBUG [RS:2;jenkins-hbase4:39791] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-24 20:10:41,459 DEBUG [RS:1;jenkins-hbase4:34371] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-24 20:10:41,459 DEBUG [RS:0;jenkins-hbase4:34483] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-24 20:10:41,467 DEBUG [RS:1;jenkins-hbase4:34371] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-24 20:10:41,467 DEBUG [RS:0;jenkins-hbase4:34483] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-24 20:10:41,467 DEBUG [RS:2;jenkins-hbase4:39791] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-24 20:10:41,467 DEBUG [RS:0;jenkins-hbase4:34483] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-24 20:10:41,467 DEBUG [RS:1;jenkins-hbase4:34371] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-24 20:10:41,468 DEBUG [RS:2;jenkins-hbase4:39791] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-24 20:10:41,471 DEBUG [RS:0;jenkins-hbase4:34483] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-24 20:10:41,471 DEBUG [RS:2;jenkins-hbase4:39791] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-24 20:10:41,471 DEBUG [RS:1;jenkins-hbase4:34371] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-24 20:10:41,473 DEBUG [RS:0;jenkins-hbase4:34483] zookeeper.ReadOnlyZKClient(139): Connect 0x507a6840 to 127.0.0.1:62881 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-24 20:10:41,473 DEBUG [RS:2;jenkins-hbase4:39791] zookeeper.ReadOnlyZKClient(139): Connect 0x683ab3ae to 127.0.0.1:62881 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-24 20:10:41,474 DEBUG [RS:1;jenkins-hbase4:34371] zookeeper.ReadOnlyZKClient(139): Connect 0x141e81d2 to 127.0.0.1:62881 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-24 20:10:41,485 DEBUG [RS:2;jenkins-hbase4:39791] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@66fd40d9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-24 20:10:41,485 DEBUG [RS:0;jenkins-hbase4:34483] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5ebd9c3f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-24 20:10:41,486 DEBUG [RS:0;jenkins-hbase4:34483] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5a820330, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-24 20:10:41,487 DEBUG [RS:2;jenkins-hbase4:39791] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@62c2d8de, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-24 20:10:41,490 DEBUG [RS:1;jenkins-hbase4:34371] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1ad97d21, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-24 20:10:41,491 DEBUG [RS:1;jenkins-hbase4:34371] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5be4771a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-24 20:10:41,522 DEBUG [RS:0;jenkins-hbase4:34483] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:34483 2023-07-24 20:10:41,524 DEBUG [RS:2;jenkins-hbase4:39791] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:39791 2023-07-24 20:10:41,525 DEBUG [RS:1;jenkins-hbase4:34371] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:34371 2023-07-24 20:10:41,531 INFO [RS:0;jenkins-hbase4:34483] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-24 20:10:41,531 INFO [RS:2;jenkins-hbase4:39791] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-24 20:10:41,535 INFO [RS:2;jenkins-hbase4:39791] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-24 20:10:41,531 INFO [RS:1;jenkins-hbase4:34371] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-24 20:10:41,535 INFO [RS:1;jenkins-hbase4:34371] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-24 20:10:41,535 DEBUG [RS:1;jenkins-hbase4:34371] regionserver.HRegionServer(1022): About to register with Master. 2023-07-24 20:10:41,535 DEBUG [RS:2;jenkins-hbase4:39791] regionserver.HRegionServer(1022): About to register with Master. 2023-07-24 20:10:41,535 INFO [RS:0;jenkins-hbase4:34483] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-24 20:10:41,536 DEBUG [RS:0;jenkins-hbase4:34483] regionserver.HRegionServer(1022): About to register with Master. 2023-07-24 20:10:41,539 INFO [RS:2;jenkins-hbase4:39791] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,39247,1690229437650 with isa=jenkins-hbase4.apache.org/172.31.14.131:39791, startcode=1690229440084 2023-07-24 20:10:41,539 INFO [RS:0;jenkins-hbase4:34483] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,39247,1690229437650 with isa=jenkins-hbase4.apache.org/172.31.14.131:34483, startcode=1690229439858 2023-07-24 20:10:41,539 INFO [RS:1;jenkins-hbase4:34371] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,39247,1690229437650 with isa=jenkins-hbase4.apache.org/172.31.14.131:34371, startcode=1690229440021 2023-07-24 20:10:41,562 DEBUG [RS:2;jenkins-hbase4:39791] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-24 20:10:41,562 DEBUG [RS:0;jenkins-hbase4:34483] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-24 20:10:41,562 DEBUG [RS:1;jenkins-hbase4:34371] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-24 20:10:41,575 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-07-24 20:10:41,625 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-24 20:10:41,633 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-24 20:10:41,634 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-24 20:10:41,634 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-24 20:10:41,636 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-24 20:10:41,636 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-24 20:10:41,636 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-24 20:10:41,636 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-24 20:10:41,636 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-24 20:10:41,637 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:41,637 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-24 20:10:41,637 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:41,637 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50003, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-07-24 20:10:41,637 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36007, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-07-24 20:10:41,637 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59881, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-07-24 20:10:41,643 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690229471643 2023-07-24 20:10:41,647 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-24 20:10:41,651 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-24 20:10:41,651 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=39247] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2832) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:579) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-24 20:10:41,655 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-07-24 20:10:41,656 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-07-24 20:10:41,659 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-24 20:10:41,664 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-24 20:10:41,664 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-24 20:10:41,664 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=39247] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2832) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:579) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-24 20:10:41,665 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-24 20:10:41,666 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-24 20:10:41,666 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=39247] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2832) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:579) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-24 20:10:41,668 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:41,676 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-24 20:10:41,678 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-24 20:10:41,679 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-24 20:10:41,683 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-24 20:10:41,683 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-24 20:10:41,686 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1690229441686,5,FailOnTimeoutGroup] 2023-07-24 20:10:41,687 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1690229441687,5,FailOnTimeoutGroup] 2023-07-24 20:10:41,687 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:41,690 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-24 20:10:41,693 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:41,693 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:41,717 DEBUG [RS:1;jenkins-hbase4:34371] regionserver.HRegionServer(2830): Master is not running yet 2023-07-24 20:10:41,717 DEBUG [RS:2;jenkins-hbase4:39791] regionserver.HRegionServer(2830): Master is not running yet 2023-07-24 20:10:41,717 WARN [RS:1;jenkins-hbase4:34371] regionserver.HRegionServer(1030): reportForDuty failed; sleeping 100 ms and then retrying. 2023-07-24 20:10:41,717 WARN [RS:2;jenkins-hbase4:39791] regionserver.HRegionServer(1030): reportForDuty failed; sleeping 100 ms and then retrying. 2023-07-24 20:10:41,717 DEBUG [RS:0;jenkins-hbase4:34483] regionserver.HRegionServer(2830): Master is not running yet 2023-07-24 20:10:41,719 WARN [RS:0;jenkins-hbase4:34483] regionserver.HRegionServer(1030): reportForDuty failed; sleeping 100 ms and then retrying. 2023-07-24 20:10:41,740 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-24 20:10:41,741 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-24 20:10:41,742 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4 2023-07-24 20:10:41,780 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:41,783 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-24 20:10:41,786 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/info 2023-07-24 20:10:41,786 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-24 20:10:41,788 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:10:41,788 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-24 20:10:41,791 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/rep_barrier 2023-07-24 20:10:41,791 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-24 20:10:41,794 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:10:41,794 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-24 20:10:41,796 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/table 2023-07-24 20:10:41,797 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-24 20:10:41,798 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:10:41,800 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740 2023-07-24 20:10:41,801 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740 2023-07-24 20:10:41,806 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-24 20:10:41,809 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-24 20:10:41,812 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 20:10:41,813 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=9994503200, jitterRate=-0.06918935477733612}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-24 20:10:41,814 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-24 20:10:41,814 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-24 20:10:41,814 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-24 20:10:41,814 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-24 20:10:41,814 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-24 20:10:41,814 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-24 20:10:41,816 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-24 20:10:41,816 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-24 20:10:41,820 INFO [RS:2;jenkins-hbase4:39791] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,39247,1690229437650 with isa=jenkins-hbase4.apache.org/172.31.14.131:39791, startcode=1690229440084 2023-07-24 20:10:41,821 INFO [RS:0;jenkins-hbase4:34483] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,39247,1690229437650 with isa=jenkins-hbase4.apache.org/172.31.14.131:34483, startcode=1690229439858 2023-07-24 20:10:41,827 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=39247] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:41,829 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,39247,1690229437650] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(787): Updating default servers. 2023-07-24 20:10:41,830 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,39247,1690229437650] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(792): Updated with servers: 1 2023-07-24 20:10:41,831 INFO [RS:1;jenkins-hbase4:34371] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,39247,1690229437650 with isa=jenkins-hbase4.apache.org/172.31.14.131:34371, startcode=1690229440021 2023-07-24 20:10:41,835 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-07-24 20:10:41,835 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-07-24 20:10:41,837 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=39247] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:41,838 DEBUG [RS:2;jenkins-hbase4:39791] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4 2023-07-24 20:10:41,838 DEBUG [RS:2;jenkins-hbase4:39791] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:37609 2023-07-24 20:10:41,838 DEBUG [RS:2;jenkins-hbase4:39791] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=36023 2023-07-24 20:10:41,838 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=39247] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,34371,1690229440021 2023-07-24 20:10:41,838 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,39247,1690229437650] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(787): Updating default servers. 2023-07-24 20:10:41,843 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,39247,1690229437650] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(792): Updated with servers: 3 2023-07-24 20:10:41,843 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,39247,1690229437650] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(787): Updating default servers. 2023-07-24 20:10:41,845 DEBUG [RS:0;jenkins-hbase4:34483] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4 2023-07-24 20:10:41,845 DEBUG [RS:1;jenkins-hbase4:34371] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4 2023-07-24 20:10:41,846 DEBUG [RS:0;jenkins-hbase4:34483] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:37609 2023-07-24 20:10:41,846 DEBUG [RS:0;jenkins-hbase4:34483] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=36023 2023-07-24 20:10:41,846 DEBUG [RS:1;jenkins-hbase4:34371] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:37609 2023-07-24 20:10:41,847 DEBUG [RS:1;jenkins-hbase4:34371] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=36023 2023-07-24 20:10:41,847 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:10:41,854 DEBUG [RS:2;jenkins-hbase4:39791] zookeeper.ZKUtil(162): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:41,855 DEBUG [RS:1;jenkins-hbase4:34371] zookeeper.ZKUtil(162): regionserver:34371-0x10198f4f3ce0002, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34371,1690229440021 2023-07-24 20:10:41,854 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-24 20:10:41,855 WARN [RS:1;jenkins-hbase4:34371] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-24 20:10:41,855 WARN [RS:2;jenkins-hbase4:39791] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-24 20:10:41,856 INFO [RS:1;jenkins-hbase4:34371] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-24 20:10:41,856 INFO [RS:2;jenkins-hbase4:39791] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-24 20:10:41,856 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,39791,1690229440084] 2023-07-24 20:10:41,857 DEBUG [RS:2;jenkins-hbase4:39791] regionserver.HRegionServer(1948): logDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:41,856 DEBUG [RS:0;jenkins-hbase4:34483] zookeeper.ZKUtil(162): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:41,857 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,34483,1690229439858] 2023-07-24 20:10:41,856 DEBUG [RS:1;jenkins-hbase4:34371] regionserver.HRegionServer(1948): logDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,34371,1690229440021 2023-07-24 20:10:41,857 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,34371,1690229440021] 2023-07-24 20:10:41,857 WARN [RS:0;jenkins-hbase4:34483] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-24 20:10:41,857 INFO [RS:0;jenkins-hbase4:34483] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-24 20:10:41,857 DEBUG [RS:0;jenkins-hbase4:34483] regionserver.HRegionServer(1948): logDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:41,879 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-24 20:10:41,882 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-07-24 20:10:41,882 DEBUG [RS:2;jenkins-hbase4:39791] zookeeper.ZKUtil(162): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34371,1690229440021 2023-07-24 20:10:41,882 DEBUG [RS:0;jenkins-hbase4:34483] zookeeper.ZKUtil(162): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34371,1690229440021 2023-07-24 20:10:41,882 DEBUG [RS:1;jenkins-hbase4:34371] zookeeper.ZKUtil(162): regionserver:34371-0x10198f4f3ce0002, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34371,1690229440021 2023-07-24 20:10:41,883 DEBUG [RS:0;jenkins-hbase4:34483] zookeeper.ZKUtil(162): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:41,883 DEBUG [RS:2;jenkins-hbase4:39791] zookeeper.ZKUtil(162): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:41,883 DEBUG [RS:1;jenkins-hbase4:34371] zookeeper.ZKUtil(162): regionserver:34371-0x10198f4f3ce0002, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:41,883 DEBUG [RS:0;jenkins-hbase4:34483] zookeeper.ZKUtil(162): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:41,884 DEBUG [RS:2;jenkins-hbase4:39791] zookeeper.ZKUtil(162): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:41,884 DEBUG [RS:1;jenkins-hbase4:34371] zookeeper.ZKUtil(162): regionserver:34371-0x10198f4f3ce0002, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:41,896 DEBUG [RS:2;jenkins-hbase4:39791] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-24 20:10:41,896 DEBUG [RS:1;jenkins-hbase4:34371] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-24 20:10:41,896 DEBUG [RS:0;jenkins-hbase4:34483] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-24 20:10:41,907 INFO [RS:2;jenkins-hbase4:39791] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-24 20:10:41,907 INFO [RS:1;jenkins-hbase4:34371] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-24 20:10:41,907 INFO [RS:0;jenkins-hbase4:34483] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-24 20:10:41,929 INFO [RS:2;jenkins-hbase4:39791] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-24 20:10:41,929 INFO [RS:0;jenkins-hbase4:34483] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-24 20:10:41,929 INFO [RS:1;jenkins-hbase4:34371] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-24 20:10:41,934 INFO [RS:0;jenkins-hbase4:34483] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-24 20:10:41,934 INFO [RS:2;jenkins-hbase4:39791] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-24 20:10:41,934 INFO [RS:1;jenkins-hbase4:34371] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-24 20:10:41,935 INFO [RS:2;jenkins-hbase4:39791] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:41,935 INFO [RS:0;jenkins-hbase4:34483] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:41,936 INFO [RS:1;jenkins-hbase4:34371] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:41,936 INFO [RS:2;jenkins-hbase4:39791] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-24 20:10:41,936 INFO [RS:1;jenkins-hbase4:34371] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-24 20:10:41,937 INFO [RS:0;jenkins-hbase4:34483] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-24 20:10:41,947 INFO [RS:1;jenkins-hbase4:34371] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:41,947 INFO [RS:0;jenkins-hbase4:34483] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:41,947 INFO [RS:2;jenkins-hbase4:39791] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:41,947 DEBUG [RS:1;jenkins-hbase4:34371] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:41,948 DEBUG [RS:0;jenkins-hbase4:34483] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:41,948 DEBUG [RS:1;jenkins-hbase4:34371] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:41,948 DEBUG [RS:0;jenkins-hbase4:34483] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:41,948 DEBUG [RS:2;jenkins-hbase4:39791] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:41,948 DEBUG [RS:0;jenkins-hbase4:34483] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:41,948 DEBUG [RS:2;jenkins-hbase4:39791] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:41,948 DEBUG [RS:0;jenkins-hbase4:34483] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:41,948 DEBUG [RS:2;jenkins-hbase4:39791] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:41,948 DEBUG [RS:1;jenkins-hbase4:34371] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:41,948 DEBUG [RS:2;jenkins-hbase4:39791] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:41,948 DEBUG [RS:0;jenkins-hbase4:34483] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:41,948 DEBUG [RS:2;jenkins-hbase4:39791] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:41,949 DEBUG [RS:0;jenkins-hbase4:34483] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-24 20:10:41,949 DEBUG [RS:2;jenkins-hbase4:39791] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-24 20:10:41,949 DEBUG [RS:0;jenkins-hbase4:34483] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:41,949 DEBUG [RS:2;jenkins-hbase4:39791] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:41,948 DEBUG [RS:1;jenkins-hbase4:34371] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:41,949 DEBUG [RS:2;jenkins-hbase4:39791] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:41,949 DEBUG [RS:0;jenkins-hbase4:34483] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:41,949 DEBUG [RS:2;jenkins-hbase4:39791] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:41,949 DEBUG [RS:0;jenkins-hbase4:34483] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:41,949 DEBUG [RS:2;jenkins-hbase4:39791] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:41,949 DEBUG [RS:0;jenkins-hbase4:34483] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:41,949 DEBUG [RS:1;jenkins-hbase4:34371] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:41,949 DEBUG [RS:1;jenkins-hbase4:34371] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-24 20:10:41,949 DEBUG [RS:1;jenkins-hbase4:34371] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:41,950 DEBUG [RS:1;jenkins-hbase4:34371] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:41,950 DEBUG [RS:1;jenkins-hbase4:34371] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:41,950 DEBUG [RS:1;jenkins-hbase4:34371] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:41,951 INFO [RS:2;jenkins-hbase4:39791] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:41,952 INFO [RS:2;jenkins-hbase4:39791] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:41,952 INFO [RS:2;jenkins-hbase4:39791] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:41,952 INFO [RS:1;jenkins-hbase4:34371] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:41,952 INFO [RS:1;jenkins-hbase4:34371] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:41,952 INFO [RS:1;jenkins-hbase4:34371] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:41,952 INFO [RS:0;jenkins-hbase4:34483] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:41,952 INFO [RS:0;jenkins-hbase4:34483] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:41,952 INFO [RS:0;jenkins-hbase4:34483] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:41,971 INFO [RS:0;jenkins-hbase4:34483] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-24 20:10:41,971 INFO [RS:2;jenkins-hbase4:39791] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-24 20:10:41,971 INFO [RS:1;jenkins-hbase4:34371] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-24 20:10:41,975 INFO [RS:0;jenkins-hbase4:34483] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34483,1690229439858-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:41,975 INFO [RS:1;jenkins-hbase4:34371] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34371,1690229440021-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:41,975 INFO [RS:2;jenkins-hbase4:39791] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39791,1690229440084-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:41,994 INFO [RS:0;jenkins-hbase4:34483] regionserver.Replication(203): jenkins-hbase4.apache.org,34483,1690229439858 started 2023-07-24 20:10:41,994 INFO [RS:0;jenkins-hbase4:34483] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,34483,1690229439858, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:34483, sessionid=0x10198f4f3ce0001 2023-07-24 20:10:41,995 DEBUG [RS:0;jenkins-hbase4:34483] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-24 20:10:41,995 DEBUG [RS:0;jenkins-hbase4:34483] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:41,995 DEBUG [RS:0;jenkins-hbase4:34483] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34483,1690229439858' 2023-07-24 20:10:41,995 DEBUG [RS:0;jenkins-hbase4:34483] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 20:10:41,995 INFO [RS:1;jenkins-hbase4:34371] regionserver.Replication(203): jenkins-hbase4.apache.org,34371,1690229440021 started 2023-07-24 20:10:41,995 INFO [RS:1;jenkins-hbase4:34371] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,34371,1690229440021, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:34371, sessionid=0x10198f4f3ce0002 2023-07-24 20:10:41,995 INFO [RS:2;jenkins-hbase4:39791] regionserver.Replication(203): jenkins-hbase4.apache.org,39791,1690229440084 started 2023-07-24 20:10:41,995 DEBUG [RS:1;jenkins-hbase4:34371] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-24 20:10:41,995 INFO [RS:2;jenkins-hbase4:39791] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,39791,1690229440084, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:39791, sessionid=0x10198f4f3ce0003 2023-07-24 20:10:41,995 DEBUG [RS:1;jenkins-hbase4:34371] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,34371,1690229440021 2023-07-24 20:10:41,996 DEBUG [RS:2;jenkins-hbase4:39791] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-24 20:10:41,996 DEBUG [RS:0;jenkins-hbase4:34483] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 20:10:41,996 DEBUG [RS:1;jenkins-hbase4:34371] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34371,1690229440021' 2023-07-24 20:10:41,998 DEBUG [RS:2;jenkins-hbase4:39791] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:41,998 DEBUG [RS:1;jenkins-hbase4:34371] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 20:10:41,998 DEBUG [RS:2;jenkins-hbase4:39791] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,39791,1690229440084' 2023-07-24 20:10:41,998 DEBUG [RS:2;jenkins-hbase4:39791] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 20:10:41,998 DEBUG [RS:0;jenkins-hbase4:34483] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-24 20:10:41,998 DEBUG [RS:1;jenkins-hbase4:34371] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 20:10:41,999 DEBUG [RS:2;jenkins-hbase4:39791] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 20:10:41,999 DEBUG [RS:0;jenkins-hbase4:34483] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-24 20:10:41,999 DEBUG [RS:0;jenkins-hbase4:34483] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:41,999 DEBUG [RS:0;jenkins-hbase4:34483] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34483,1690229439858' 2023-07-24 20:10:41,999 DEBUG [RS:0;jenkins-hbase4:34483] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-24 20:10:41,999 DEBUG [RS:1;jenkins-hbase4:34371] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-24 20:10:41,999 DEBUG [RS:2;jenkins-hbase4:39791] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-24 20:10:41,999 DEBUG [RS:1;jenkins-hbase4:34371] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-24 20:10:41,999 DEBUG [RS:1;jenkins-hbase4:34371] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,34371,1690229440021 2023-07-24 20:10:41,999 DEBUG [RS:2;jenkins-hbase4:39791] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-24 20:10:41,999 DEBUG [RS:2;jenkins-hbase4:39791] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:41,999 DEBUG [RS:2;jenkins-hbase4:39791] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,39791,1690229440084' 2023-07-24 20:10:41,999 DEBUG [RS:0;jenkins-hbase4:34483] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-24 20:10:41,999 DEBUG [RS:1;jenkins-hbase4:34371] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34371,1690229440021' 2023-07-24 20:10:41,999 DEBUG [RS:2;jenkins-hbase4:39791] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-24 20:10:42,000 DEBUG [RS:1;jenkins-hbase4:34371] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-24 20:10:42,000 DEBUG [RS:0;jenkins-hbase4:34483] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-24 20:10:42,000 INFO [RS:0;jenkins-hbase4:34483] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-24 20:10:42,000 INFO [RS:0;jenkins-hbase4:34483] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-24 20:10:42,000 DEBUG [RS:2;jenkins-hbase4:39791] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-24 20:10:42,000 DEBUG [RS:1;jenkins-hbase4:34371] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-24 20:10:42,001 DEBUG [RS:1;jenkins-hbase4:34371] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-24 20:10:42,001 DEBUG [RS:2;jenkins-hbase4:39791] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-24 20:10:42,001 INFO [RS:2;jenkins-hbase4:39791] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-24 20:10:42,001 INFO [RS:2;jenkins-hbase4:39791] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-24 20:10:42,001 INFO [RS:1;jenkins-hbase4:34371] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-24 20:10:42,001 INFO [RS:1;jenkins-hbase4:34371] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-24 20:10:42,034 DEBUG [jenkins-hbase4:39247] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-07-24 20:10:42,053 DEBUG [jenkins-hbase4:39247] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-24 20:10:42,055 DEBUG [jenkins-hbase4:39247] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 20:10:42,055 DEBUG [jenkins-hbase4:39247] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 20:10:42,055 DEBUG [jenkins-hbase4:39247] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 20:10:42,055 DEBUG [jenkins-hbase4:39247] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 20:10:42,060 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,34371,1690229440021, state=OPENING 2023-07-24 20:10:42,069 DEBUG [PEWorker-5] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-07-24 20:10:42,070 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 20:10:42,071 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-24 20:10:42,075 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,34371,1690229440021}] 2023-07-24 20:10:42,113 INFO [RS:2;jenkins-hbase4:39791] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C39791%2C1690229440084, suffix=, logDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,39791,1690229440084, archiveDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/oldWALs, maxLogs=32 2023-07-24 20:10:42,113 INFO [RS:0;jenkins-hbase4:34483] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34483%2C1690229439858, suffix=, logDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,34483,1690229439858, archiveDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/oldWALs, maxLogs=32 2023-07-24 20:10:42,113 INFO [RS:1;jenkins-hbase4:34371] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34371%2C1690229440021, suffix=, logDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,34371,1690229440021, archiveDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/oldWALs, maxLogs=32 2023-07-24 20:10:42,142 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK] 2023-07-24 20:10:42,143 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK] 2023-07-24 20:10:42,145 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK] 2023-07-24 20:10:42,151 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK] 2023-07-24 20:10:42,151 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK] 2023-07-24 20:10:42,151 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK] 2023-07-24 20:10:42,152 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK] 2023-07-24 20:10:42,152 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK] 2023-07-24 20:10:42,153 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK] 2023-07-24 20:10:42,169 INFO [RS:1;jenkins-hbase4:34371] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,34371,1690229440021/jenkins-hbase4.apache.org%2C34371%2C1690229440021.1690229442118 2023-07-24 20:10:42,169 DEBUG [RS:1;jenkins-hbase4:34371] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK], DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK], DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK]] 2023-07-24 20:10:42,171 INFO [RS:0;jenkins-hbase4:34483] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,34483,1690229439858/jenkins-hbase4.apache.org%2C34483%2C1690229439858.1690229442118 2023-07-24 20:10:42,171 INFO [RS:2;jenkins-hbase4:39791] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,39791,1690229440084/jenkins-hbase4.apache.org%2C39791%2C1690229440084.1690229442118 2023-07-24 20:10:42,178 DEBUG [RS:0;jenkins-hbase4:34483] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK], DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK], DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK]] 2023-07-24 20:10:42,178 DEBUG [RS:2;jenkins-hbase4:39791] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK], DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK], DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK]] 2023-07-24 20:10:42,232 WARN [ReadOnlyZKClient-127.0.0.1:62881@0x4c458dfb] client.ZKConnectionRegistry(168): Meta region is in state OPENING 2023-07-24 20:10:42,258 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,34371,1690229440021 2023-07-24 20:10:42,260 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-24 20:10:42,264 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48618, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-24 20:10:42,267 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,39247,1690229437650] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-24 20:10:42,273 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48634, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-24 20:10:42,273 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34371] ipc.CallRunner(144): callId: 1 service: ClientService methodName: Get size: 88 connection: 172.31.14.131:48634 deadline: 1690229502273, exception=org.apache.hadoop.hbase.NotServingRegionException: hbase:meta,,1 is not online on jenkins-hbase4.apache.org,34371,1690229440021 2023-07-24 20:10:42,280 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-24 20:10:42,280 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-24 20:10:42,284 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34371%2C1690229440021.meta, suffix=.meta, logDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,34371,1690229440021, archiveDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/oldWALs, maxLogs=32 2023-07-24 20:10:42,307 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK] 2023-07-24 20:10:42,312 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK] 2023-07-24 20:10:42,314 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK] 2023-07-24 20:10:42,318 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,34371,1690229440021/jenkins-hbase4.apache.org%2C34371%2C1690229440021.meta.1690229442285.meta 2023-07-24 20:10:42,319 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK], DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK], DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK]] 2023-07-24 20:10:42,319 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-24 20:10:42,321 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-24 20:10:42,324 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-24 20:10:42,326 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-24 20:10:42,332 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-24 20:10:42,332 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:42,332 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-24 20:10:42,332 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-24 20:10:42,334 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-24 20:10:42,336 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/info 2023-07-24 20:10:42,336 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/info 2023-07-24 20:10:42,337 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-24 20:10:42,338 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:10:42,338 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-24 20:10:42,339 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/rep_barrier 2023-07-24 20:10:42,339 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/rep_barrier 2023-07-24 20:10:42,340 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-24 20:10:42,341 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:10:42,341 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-24 20:10:42,342 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/table 2023-07-24 20:10:42,342 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/table 2023-07-24 20:10:42,343 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-24 20:10:42,343 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:10:42,345 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740 2023-07-24 20:10:42,348 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740 2023-07-24 20:10:42,351 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-24 20:10:42,354 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-24 20:10:42,356 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=10605801920, jitterRate=-0.01225772500038147}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-24 20:10:42,356 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-24 20:10:42,368 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1690229442254 2023-07-24 20:10:42,388 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-24 20:10:42,389 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-24 20:10:42,389 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,34371,1690229440021, state=OPEN 2023-07-24 20:10:42,392 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-07-24 20:10:42,392 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-24 20:10:42,396 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-07-24 20:10:42,396 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,34371,1690229440021 in 317 msec 2023-07-24 20:10:42,404 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-07-24 20:10:42,404 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 543 msec 2023-07-24 20:10:42,410 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 943 msec 2023-07-24 20:10:42,410 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1690229442410, completionTime=-1 2023-07-24 20:10:42,411 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-07-24 20:10:42,411 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 20:10:42,469 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 20:10:42,469 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690229502469 2023-07-24 20:10:42,469 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690229562469 2023-07-24 20:10:42,469 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 58 msec 2023-07-24 20:10:42,492 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39247,1690229437650-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:42,492 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39247,1690229437650-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:42,492 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39247,1690229437650-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:42,494 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:39247, period=300000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:42,494 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:42,501 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-07-24 20:10:42,517 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-07-24 20:10:42,518 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-24 20:10:42,530 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-07-24 20:10:42,533 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-07-24 20:10:42,536 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-24 20:10:42,556 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/hbase/namespace/324410cf2a29c2dca1a19e8570799d47 2023-07-24 20:10:42,559 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/hbase/namespace/324410cf2a29c2dca1a19e8570799d47 empty. 2023-07-24 20:10:42,560 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/hbase/namespace/324410cf2a29c2dca1a19e8570799d47 2023-07-24 20:10:42,560 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-07-24 20:10:42,600 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-07-24 20:10:42,602 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => 324410cf2a29c2dca1a19e8570799d47, NAME => 'hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp 2023-07-24 20:10:42,619 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:42,619 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing 324410cf2a29c2dca1a19e8570799d47, disabling compactions & flushes 2023-07-24 20:10:42,620 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. 2023-07-24 20:10:42,620 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. 2023-07-24 20:10:42,620 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. after waiting 0 ms 2023-07-24 20:10:42,620 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. 2023-07-24 20:10:42,620 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. 2023-07-24 20:10:42,620 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for 324410cf2a29c2dca1a19e8570799d47: 2023-07-24 20:10:42,624 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-07-24 20:10:42,648 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690229442628"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229442628"}]},"ts":"1690229442628"} 2023-07-24 20:10:42,684 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-24 20:10:42,687 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-24 20:10:42,692 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690229442687"}]},"ts":"1690229442687"} 2023-07-24 20:10:42,699 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-07-24 20:10:42,703 DEBUG [PEWorker-5] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-24 20:10:42,703 DEBUG [PEWorker-5] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 20:10:42,703 DEBUG [PEWorker-5] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 20:10:42,703 DEBUG [PEWorker-5] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 20:10:42,703 DEBUG [PEWorker-5] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 20:10:42,705 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=324410cf2a29c2dca1a19e8570799d47, ASSIGN}] 2023-07-24 20:10:42,708 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=324410cf2a29c2dca1a19e8570799d47, ASSIGN 2023-07-24 20:10:42,711 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=324410cf2a29c2dca1a19e8570799d47, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39791,1690229440084; forceNewPlan=false, retain=false 2023-07-24 20:10:42,794 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,39247,1690229437650] master.HMaster(2148): Client=null/null create 'hbase:rsgroup', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|', METADATA => {'SPLIT_POLICY' => 'org.apache.hadoop.hbase.regionserver.DisabledRegionSplitPolicy'}}}, {NAME => 'm', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-24 20:10:42,796 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,39247,1690229437650] procedure2.ProcedureExecutor(1029): Stored pid=6, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:rsgroup 2023-07-24 20:10:42,799 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=6, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:rsgroup execute state=CREATE_TABLE_PRE_OPERATION 2023-07-24 20:10:42,801 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=6, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:rsgroup execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-24 20:10:42,805 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/hbase/rsgroup/ec4f892f95beff2c1aa7863da43a18fa 2023-07-24 20:10:42,806 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/hbase/rsgroup/ec4f892f95beff2c1aa7863da43a18fa empty. 2023-07-24 20:10:42,807 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/hbase/rsgroup/ec4f892f95beff2c1aa7863da43a18fa 2023-07-24 20:10:42,807 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived hbase:rsgroup regions 2023-07-24 20:10:42,837 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/hbase/rsgroup/.tabledesc/.tableinfo.0000000001 2023-07-24 20:10:42,839 INFO [RegionOpenAndInit-hbase:rsgroup-pool-0] regionserver.HRegion(7675): creating {ENCODED => ec4f892f95beff2c1aa7863da43a18fa, NAME => 'hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:rsgroup', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|', METADATA => {'SPLIT_POLICY' => 'org.apache.hadoop.hbase.regionserver.DisabledRegionSplitPolicy'}}}, {NAME => 'm', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp 2023-07-24 20:10:42,858 DEBUG [RegionOpenAndInit-hbase:rsgroup-pool-0] regionserver.HRegion(866): Instantiated hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:42,859 DEBUG [RegionOpenAndInit-hbase:rsgroup-pool-0] regionserver.HRegion(1604): Closing ec4f892f95beff2c1aa7863da43a18fa, disabling compactions & flushes 2023-07-24 20:10:42,859 INFO [RegionOpenAndInit-hbase:rsgroup-pool-0] regionserver.HRegion(1626): Closing region hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. 2023-07-24 20:10:42,859 DEBUG [RegionOpenAndInit-hbase:rsgroup-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. 2023-07-24 20:10:42,859 DEBUG [RegionOpenAndInit-hbase:rsgroup-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. after waiting 0 ms 2023-07-24 20:10:42,859 DEBUG [RegionOpenAndInit-hbase:rsgroup-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. 2023-07-24 20:10:42,859 INFO [RegionOpenAndInit-hbase:rsgroup-pool-0] regionserver.HRegion(1838): Closed hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. 2023-07-24 20:10:42,859 DEBUG [RegionOpenAndInit-hbase:rsgroup-pool-0] regionserver.HRegion(1558): Region close journal for ec4f892f95beff2c1aa7863da43a18fa: 2023-07-24 20:10:42,862 INFO [jenkins-hbase4:39247] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-24 20:10:42,863 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=324410cf2a29c2dca1a19e8570799d47, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:42,864 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690229442863"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229442863"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229442863"}]},"ts":"1690229442863"} 2023-07-24 20:10:42,864 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=6, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:rsgroup execute state=CREATE_TABLE_ADD_TO_META 2023-07-24 20:10:42,866 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa.","families":{"info":[{"qualifier":"regioninfo","vlen":39,"tag":[],"timestamp":"1690229442865"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229442865"}]},"ts":"1690229442865"} 2023-07-24 20:10:42,870 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=7, ppid=5, state=RUNNABLE; OpenRegionProcedure 324410cf2a29c2dca1a19e8570799d47, server=jenkins-hbase4.apache.org,39791,1690229440084}] 2023-07-24 20:10:42,873 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-24 20:10:42,876 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=6, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:rsgroup execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-24 20:10:42,876 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:rsgroup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690229442876"}]},"ts":"1690229442876"} 2023-07-24 20:10:42,878 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:rsgroup, state=ENABLING in hbase:meta 2023-07-24 20:10:42,883 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-24 20:10:42,884 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 20:10:42,884 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 20:10:42,884 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 20:10:42,884 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 20:10:42,884 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=8, ppid=6, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:rsgroup, region=ec4f892f95beff2c1aa7863da43a18fa, ASSIGN}] 2023-07-24 20:10:42,886 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=8, ppid=6, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:rsgroup, region=ec4f892f95beff2c1aa7863da43a18fa, ASSIGN 2023-07-24 20:10:42,887 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=8, ppid=6, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:rsgroup, region=ec4f892f95beff2c1aa7863da43a18fa, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39791,1690229440084; forceNewPlan=false, retain=false 2023-07-24 20:10:43,026 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:43,026 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-24 20:10:43,030 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37852, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-24 20:10:43,035 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. 2023-07-24 20:10:43,035 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 324410cf2a29c2dca1a19e8570799d47, NAME => 'hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47.', STARTKEY => '', ENDKEY => ''} 2023-07-24 20:10:43,036 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 324410cf2a29c2dca1a19e8570799d47 2023-07-24 20:10:43,036 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:43,036 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 324410cf2a29c2dca1a19e8570799d47 2023-07-24 20:10:43,036 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 324410cf2a29c2dca1a19e8570799d47 2023-07-24 20:10:43,038 INFO [jenkins-hbase4:39247] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-24 20:10:43,039 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=8 updating hbase:meta row=ec4f892f95beff2c1aa7863da43a18fa, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:43,039 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa.","families":{"info":[{"qualifier":"regioninfo","vlen":39,"tag":[],"timestamp":"1690229443038"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229443038"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229443038"}]},"ts":"1690229443038"} 2023-07-24 20:10:43,039 INFO [StoreOpener-324410cf2a29c2dca1a19e8570799d47-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 324410cf2a29c2dca1a19e8570799d47 2023-07-24 20:10:43,042 DEBUG [StoreOpener-324410cf2a29c2dca1a19e8570799d47-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/namespace/324410cf2a29c2dca1a19e8570799d47/info 2023-07-24 20:10:43,042 DEBUG [StoreOpener-324410cf2a29c2dca1a19e8570799d47-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/namespace/324410cf2a29c2dca1a19e8570799d47/info 2023-07-24 20:10:43,042 INFO [StoreOpener-324410cf2a29c2dca1a19e8570799d47-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 324410cf2a29c2dca1a19e8570799d47 columnFamilyName info 2023-07-24 20:10:43,043 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=9, ppid=8, state=RUNNABLE; OpenRegionProcedure ec4f892f95beff2c1aa7863da43a18fa, server=jenkins-hbase4.apache.org,39791,1690229440084}] 2023-07-24 20:10:43,043 INFO [StoreOpener-324410cf2a29c2dca1a19e8570799d47-1] regionserver.HStore(310): Store=324410cf2a29c2dca1a19e8570799d47/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:10:43,046 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/namespace/324410cf2a29c2dca1a19e8570799d47 2023-07-24 20:10:43,046 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/namespace/324410cf2a29c2dca1a19e8570799d47 2023-07-24 20:10:43,051 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 324410cf2a29c2dca1a19e8570799d47 2023-07-24 20:10:43,055 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/namespace/324410cf2a29c2dca1a19e8570799d47/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 20:10:43,055 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 324410cf2a29c2dca1a19e8570799d47; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=9438844800, jitterRate=-0.12093907594680786}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 20:10:43,056 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 324410cf2a29c2dca1a19e8570799d47: 2023-07-24 20:10:43,057 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47., pid=7, masterSystemTime=1690229443026 2023-07-24 20:10:43,062 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. 2023-07-24 20:10:43,062 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. 2023-07-24 20:10:43,063 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=324410cf2a29c2dca1a19e8570799d47, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:43,064 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690229443063"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690229443063"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690229443063"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690229443063"}]},"ts":"1690229443063"} 2023-07-24 20:10:43,071 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=7, resume processing ppid=5 2023-07-24 20:10:43,071 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=7, ppid=5, state=SUCCESS; OpenRegionProcedure 324410cf2a29c2dca1a19e8570799d47, server=jenkins-hbase4.apache.org,39791,1690229440084 in 197 msec 2023-07-24 20:10:43,074 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-07-24 20:10:43,075 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=324410cf2a29c2dca1a19e8570799d47, ASSIGN in 366 msec 2023-07-24 20:10:43,076 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-24 20:10:43,077 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690229443076"}]},"ts":"1690229443076"} 2023-07-24 20:10:43,079 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-07-24 20:10:43,083 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-07-24 20:10:43,124 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 564 msec 2023-07-24 20:10:43,133 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-07-24 20:10:43,135 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-07-24 20:10:43,135 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 20:10:43,159 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-24 20:10:43,161 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37854, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-24 20:10:43,177 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=10, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-07-24 20:10:43,194 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-07-24 20:10:43,204 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. 2023-07-24 20:10:43,204 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ec4f892f95beff2c1aa7863da43a18fa, NAME => 'hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa.', STARTKEY => '', ENDKEY => ''} 2023-07-24 20:10:43,204 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-24 20:10:43,204 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. service=MultiRowMutationService 2023-07-24 20:10:43,205 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:rsgroup successfully. 2023-07-24 20:10:43,205 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table rsgroup ec4f892f95beff2c1aa7863da43a18fa 2023-07-24 20:10:43,205 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:43,205 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ec4f892f95beff2c1aa7863da43a18fa 2023-07-24 20:10:43,205 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ec4f892f95beff2c1aa7863da43a18fa 2023-07-24 20:10:43,207 INFO [StoreOpener-ec4f892f95beff2c1aa7863da43a18fa-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family m of region ec4f892f95beff2c1aa7863da43a18fa 2023-07-24 20:10:43,210 DEBUG [StoreOpener-ec4f892f95beff2c1aa7863da43a18fa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/rsgroup/ec4f892f95beff2c1aa7863da43a18fa/m 2023-07-24 20:10:43,210 DEBUG [StoreOpener-ec4f892f95beff2c1aa7863da43a18fa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/rsgroup/ec4f892f95beff2c1aa7863da43a18fa/m 2023-07-24 20:10:43,210 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=10, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 42 msec 2023-07-24 20:10:43,211 INFO [StoreOpener-ec4f892f95beff2c1aa7863da43a18fa-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ec4f892f95beff2c1aa7863da43a18fa columnFamilyName m 2023-07-24 20:10:43,211 INFO [StoreOpener-ec4f892f95beff2c1aa7863da43a18fa-1] regionserver.HStore(310): Store=ec4f892f95beff2c1aa7863da43a18fa/m, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:10:43,213 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/rsgroup/ec4f892f95beff2c1aa7863da43a18fa 2023-07-24 20:10:43,214 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/rsgroup/ec4f892f95beff2c1aa7863da43a18fa 2023-07-24 20:10:43,219 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ec4f892f95beff2c1aa7863da43a18fa 2023-07-24 20:10:43,222 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=11, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-07-24 20:10:43,225 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/rsgroup/ec4f892f95beff2c1aa7863da43a18fa/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 20:10:43,226 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ec4f892f95beff2c1aa7863da43a18fa; next sequenceid=2; org.apache.hadoop.hbase.regionserver.DisabledRegionSplitPolicy@38354ae8, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 20:10:43,226 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ec4f892f95beff2c1aa7863da43a18fa: 2023-07-24 20:10:43,226 DEBUG [PEWorker-1] procedure.MasterProcedureScheduler(526): NAMESPACE 'hbase', shared lock count=1 2023-07-24 20:10:43,226 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=11, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-07-24 20:10:43,227 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa., pid=9, masterSystemTime=1690229443198 2023-07-24 20:10:43,230 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. 2023-07-24 20:10:43,230 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. 2023-07-24 20:10:43,231 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=8 updating hbase:meta row=ec4f892f95beff2c1aa7863da43a18fa, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:43,231 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa.","families":{"info":[{"qualifier":"regioninfo","vlen":39,"tag":[],"timestamp":"1690229443230"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690229443230"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690229443230"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690229443230"}]},"ts":"1690229443230"} 2023-07-24 20:10:43,238 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=9, resume processing ppid=8 2023-07-24 20:10:43,238 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=9, ppid=8, state=SUCCESS; OpenRegionProcedure ec4f892f95beff2c1aa7863da43a18fa, server=jenkins-hbase4.apache.org,39791,1690229440084 in 190 msec 2023-07-24 20:10:43,244 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=8, resume processing ppid=6 2023-07-24 20:10:43,245 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=8, ppid=6, state=SUCCESS; TransitRegionStateProcedure table=hbase:rsgroup, region=ec4f892f95beff2c1aa7863da43a18fa, ASSIGN in 354 msec 2023-07-24 20:10:43,255 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-07-24 20:10:43,276 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=11, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 47 msec 2023-07-24 20:10:43,283 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=6, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:rsgroup execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-24 20:10:43,284 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:rsgroup","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690229443284"}]},"ts":"1690229443284"} 2023-07-24 20:10:43,287 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:rsgroup, state=ENABLED in hbase:meta 2023-07-24 20:10:43,290 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=6, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:rsgroup execute state=CREATE_TABLE_POST_OPERATION 2023-07-24 20:10:43,293 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=6, state=SUCCESS; CreateTableProcedure table=hbase:rsgroup in 497 msec 2023-07-24 20:10:43,298 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-07-24 20:10:43,301 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-07-24 20:10:43,301 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 3.065sec 2023-07-24 20:10:43,305 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-24 20:10:43,306 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-24 20:10:43,306 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-24 20:10:43,308 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39247,1690229437650-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-24 20:10:43,309 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,39247,1690229437650-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-24 20:10:43,321 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-24 20:10:43,331 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,39247,1690229437650] rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker(840): RSGroup table=hbase:rsgroup is online, refreshing cached information 2023-07-24 20:10:43,331 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,39247,1690229437650] rsgroup.RSGroupInfoManagerImpl(534): Refreshing in Online mode. 2023-07-24 20:10:43,359 DEBUG [Listener at localhost/44145] zookeeper.ReadOnlyZKClient(139): Connect 0x6ecbabf0 to 127.0.0.1:62881 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-24 20:10:43,367 DEBUG [Listener at localhost/44145] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5a77b5fa, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-24 20:10:43,383 DEBUG [hconnection-0x4636fd6e-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-24 20:10:43,400 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48640, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-24 20:10:43,406 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 20:10:43,406 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,39247,1690229437650] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:43,411 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,39247,1690229437650] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 2 2023-07-24 20:10:43,412 INFO [Listener at localhost/44145] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,39247,1690229437650 2023-07-24 20:10:43,413 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:10:43,417 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,39247,1690229437650] rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker(823): GroupBasedLoadBalancer is now online 2023-07-24 20:10:43,425 DEBUG [Listener at localhost/44145] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-07-24 20:10:43,431 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36610, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-07-24 20:10:43,447 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-07-24 20:10:43,447 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 20:10:43,450 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-07-24 20:10:43,456 DEBUG [Listener at localhost/44145] zookeeper.ReadOnlyZKClient(139): Connect 0x01c3dc8e to 127.0.0.1:62881 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-24 20:10:43,462 DEBUG [Listener at localhost/44145] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@42c3f966, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-24 20:10:43,463 INFO [Listener at localhost/44145] zookeeper.RecoverableZooKeeper(93): Process identifier=VerifyingRSGroupAdminClient connecting to ZooKeeper ensemble=127.0.0.1:62881 2023-07-24 20:10:43,467 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): VerifyingRSGroupAdminClient0x0, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-24 20:10:43,476 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): VerifyingRSGroupAdminClient-0x10198f4f3ce000a connected 2023-07-24 20:10:43,500 INFO [Listener at localhost/44145] hbase.ResourceChecker(147): before: rsgroup.TestRSGroupsBasics#testClearNotProcessedDeadServer Thread=422, OpenFileDescriptor=673, MaxFileDescriptor=60000, SystemLoadAverage=393, ProcessCount=176, AvailableMemoryMB=6070 2023-07-24 20:10:43,502 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase(132): testClearNotProcessedDeadServer 2023-07-24 20:10:43,530 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:43,532 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:43,597 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase(152): Restoring servers: 1 2023-07-24 20:10:43,610 INFO [Listener at localhost/44145] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-24 20:10:43,610 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 20:10:43,610 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-24 20:10:43,610 INFO [Listener at localhost/44145] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-24 20:10:43,610 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 20:10:43,611 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-24 20:10:43,611 INFO [Listener at localhost/44145] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-24 20:10:43,615 INFO [Listener at localhost/44145] ipc.NettyRpcServer(120): Bind to /172.31.14.131:34917 2023-07-24 20:10:43,616 INFO [Listener at localhost/44145] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-24 20:10:43,621 DEBUG [Listener at localhost/44145] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-24 20:10:43,623 INFO [Listener at localhost/44145] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:10:43,628 INFO [Listener at localhost/44145] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:10:43,630 INFO [Listener at localhost/44145] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:34917 connecting to ZooKeeper ensemble=127.0.0.1:62881 2023-07-24 20:10:43,635 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(162): regionserver:349170x0, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-24 20:10:43,637 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(162): regionserver:349170x0, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/running 2023-07-24 20:10:43,638 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): regionserver:349170x0, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-24 20:10:43,639 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:349170x0, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-24 20:10:43,643 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34917 2023-07-24 20:10:43,643 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:34917-0x10198f4f3ce000b connected 2023-07-24 20:10:43,645 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34917 2023-07-24 20:10:43,646 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34917 2023-07-24 20:10:43,648 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34917 2023-07-24 20:10:43,648 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34917 2023-07-24 20:10:43,650 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2023-07-24 20:10:43,650 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter) 2023-07-24 20:10:43,650 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'securityheaders' (class=org.apache.hadoop.hbase.http.SecurityHeadersFilter) 2023-07-24 20:10:43,651 INFO [Listener at localhost/44145] http.HttpServer(879): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context regionserver 2023-07-24 20:10:43,651 INFO [Listener at localhost/44145] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2023-07-24 20:10:43,651 INFO [Listener at localhost/44145] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2023-07-24 20:10:43,651 INFO [Listener at localhost/44145] http.HttpServer(783): ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint. 2023-07-24 20:10:43,652 INFO [Listener at localhost/44145] http.HttpServer(1146): Jetty bound to port 37561 2023-07-24 20:10:43,652 INFO [Listener at localhost/44145] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-24 20:10:43,655 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:10:43,655 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@42f42ab7{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/hadoop.log.dir/,AVAILABLE} 2023-07-24 20:10:43,655 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:10:43,656 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@8b44a53{static,/static,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/static/,AVAILABLE} 2023-07-24 20:10:43,664 INFO [Listener at localhost/44145] webapp.StandardDescriptorProcessor(277): NO JSP Support for /, did not find org.apache.hbase.thirdparty.org.eclipse.jetty.jsp.JettyJspServlet 2023-07-24 20:10:43,664 INFO [Listener at localhost/44145] session.DefaultSessionIdManager(334): DefaultSessionIdManager workerName=node0 2023-07-24 20:10:43,665 INFO [Listener at localhost/44145] session.DefaultSessionIdManager(339): No SessionScavenger set, using defaults 2023-07-24 20:10:43,665 INFO [Listener at localhost/44145] session.HouseKeeper(132): node0 Scavenging every 600000ms 2023-07-24 20:10:43,667 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:10:43,669 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.w.WebAppContext@2f83be5f{regionserver,/,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver/,AVAILABLE}{file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver} 2023-07-24 20:10:43,670 INFO [Listener at localhost/44145] server.AbstractConnector(333): Started ServerConnector@41896d02{HTTP/1.1, (http/1.1)}{0.0.0.0:37561} 2023-07-24 20:10:43,671 INFO [Listener at localhost/44145] server.Server(415): Started @12238ms 2023-07-24 20:10:43,676 INFO [RS:3;jenkins-hbase4:34917] regionserver.HRegionServer(951): ClusterId : dc54b028-3544-4133-8f0e-d550a19bc790 2023-07-24 20:10:43,677 DEBUG [RS:3;jenkins-hbase4:34917] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-24 20:10:43,681 DEBUG [RS:3;jenkins-hbase4:34917] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-24 20:10:43,681 DEBUG [RS:3;jenkins-hbase4:34917] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-24 20:10:43,684 DEBUG [RS:3;jenkins-hbase4:34917] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-24 20:10:43,685 DEBUG [RS:3;jenkins-hbase4:34917] zookeeper.ReadOnlyZKClient(139): Connect 0x01c5fe6f to 127.0.0.1:62881 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-24 20:10:43,694 DEBUG [RS:3;jenkins-hbase4:34917] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@f020027, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-24 20:10:43,694 DEBUG [RS:3;jenkins-hbase4:34917] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@b3d951f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-24 20:10:43,707 DEBUG [RS:3;jenkins-hbase4:34917] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:3;jenkins-hbase4:34917 2023-07-24 20:10:43,707 INFO [RS:3;jenkins-hbase4:34917] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-24 20:10:43,707 INFO [RS:3;jenkins-hbase4:34917] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-24 20:10:43,707 DEBUG [RS:3;jenkins-hbase4:34917] regionserver.HRegionServer(1022): About to register with Master. 2023-07-24 20:10:43,708 INFO [RS:3;jenkins-hbase4:34917] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,39247,1690229437650 with isa=jenkins-hbase4.apache.org/172.31.14.131:34917, startcode=1690229443609 2023-07-24 20:10:43,708 DEBUG [RS:3;jenkins-hbase4:34917] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-24 20:10:43,715 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59739, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.3 (auth:SIMPLE), service=RegionServerStatusService 2023-07-24 20:10:43,715 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=39247] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:43,715 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,39247,1690229437650] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(787): Updating default servers. 2023-07-24 20:10:43,716 DEBUG [RS:3;jenkins-hbase4:34917] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4 2023-07-24 20:10:43,716 DEBUG [RS:3;jenkins-hbase4:34917] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:37609 2023-07-24 20:10:43,716 DEBUG [RS:3;jenkins-hbase4:34917] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=36023 2023-07-24 20:10:43,721 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:10:43,721 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:10:43,721 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:10:43,721 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,39247,1690229437650] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:43,722 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,39247,1690229437650] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 2 2023-07-24 20:10:43,722 DEBUG [RS:3;jenkins-hbase4:34917] zookeeper.ZKUtil(162): regionserver:34917-0x10198f4f3ce000b, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:43,722 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,34917,1690229443609] 2023-07-24 20:10:43,722 WARN [RS:3;jenkins-hbase4:34917] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-24 20:10:43,722 INFO [RS:3;jenkins-hbase4:34917] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-24 20:10:43,722 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34371,1690229440021 2023-07-24 20:10:43,722 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34371,1690229440021 2023-07-24 20:10:43,722 DEBUG [RS:3;jenkins-hbase4:34917] regionserver.HRegionServer(1948): logDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:43,726 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34371-0x10198f4f3ce0002, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:10:43,732 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:43,732 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,39247,1690229437650] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(792): Updated with servers: 4 2023-07-24 20:10:43,733 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:43,733 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:43,734 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:43,734 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34371-0x10198f4f3ce0002, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34371,1690229440021 2023-07-24 20:10:43,734 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:43,734 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:43,735 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34371-0x10198f4f3ce0002, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:43,736 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34371-0x10198f4f3ce0002, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:43,737 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34371-0x10198f4f3ce0002, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:43,738 DEBUG [RS:3;jenkins-hbase4:34917] zookeeper.ZKUtil(162): regionserver:34917-0x10198f4f3ce000b, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34371,1690229440021 2023-07-24 20:10:43,739 DEBUG [RS:3;jenkins-hbase4:34917] zookeeper.ZKUtil(162): regionserver:34917-0x10198f4f3ce000b, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:43,739 DEBUG [RS:3;jenkins-hbase4:34917] zookeeper.ZKUtil(162): regionserver:34917-0x10198f4f3ce000b, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:43,740 DEBUG [RS:3;jenkins-hbase4:34917] zookeeper.ZKUtil(162): regionserver:34917-0x10198f4f3ce000b, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:43,741 DEBUG [RS:3;jenkins-hbase4:34917] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-24 20:10:43,741 INFO [RS:3;jenkins-hbase4:34917] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-24 20:10:43,744 INFO [RS:3;jenkins-hbase4:34917] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-24 20:10:43,747 INFO [RS:3;jenkins-hbase4:34917] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-24 20:10:43,747 INFO [RS:3;jenkins-hbase4:34917] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:43,747 INFO [RS:3;jenkins-hbase4:34917] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-24 20:10:43,749 INFO [RS:3;jenkins-hbase4:34917] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:43,749 DEBUG [RS:3;jenkins-hbase4:34917] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:43,750 DEBUG [RS:3;jenkins-hbase4:34917] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:43,750 DEBUG [RS:3;jenkins-hbase4:34917] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:43,750 DEBUG [RS:3;jenkins-hbase4:34917] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:43,750 DEBUG [RS:3;jenkins-hbase4:34917] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:43,750 DEBUG [RS:3;jenkins-hbase4:34917] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-24 20:10:43,750 DEBUG [RS:3;jenkins-hbase4:34917] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:43,750 DEBUG [RS:3;jenkins-hbase4:34917] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:43,750 DEBUG [RS:3;jenkins-hbase4:34917] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:43,750 DEBUG [RS:3;jenkins-hbase4:34917] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:43,751 INFO [RS:3;jenkins-hbase4:34917] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:43,751 INFO [RS:3;jenkins-hbase4:34917] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:43,751 INFO [RS:3;jenkins-hbase4:34917] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:43,763 INFO [RS:3;jenkins-hbase4:34917] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-24 20:10:43,763 INFO [RS:3;jenkins-hbase4:34917] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34917,1690229443609-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:43,776 INFO [RS:3;jenkins-hbase4:34917] regionserver.Replication(203): jenkins-hbase4.apache.org,34917,1690229443609 started 2023-07-24 20:10:43,776 INFO [RS:3;jenkins-hbase4:34917] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,34917,1690229443609, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:34917, sessionid=0x10198f4f3ce000b 2023-07-24 20:10:43,776 DEBUG [RS:3;jenkins-hbase4:34917] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-24 20:10:43,776 DEBUG [RS:3;jenkins-hbase4:34917] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:43,776 DEBUG [RS:3;jenkins-hbase4:34917] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34917,1690229443609' 2023-07-24 20:10:43,776 DEBUG [RS:3;jenkins-hbase4:34917] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 20:10:43,776 DEBUG [RS:3;jenkins-hbase4:34917] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 20:10:43,777 DEBUG [RS:3;jenkins-hbase4:34917] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-24 20:10:43,777 DEBUG [RS:3;jenkins-hbase4:34917] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-24 20:10:43,777 DEBUG [RS:3;jenkins-hbase4:34917] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:43,777 DEBUG [RS:3;jenkins-hbase4:34917] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34917,1690229443609' 2023-07-24 20:10:43,777 DEBUG [RS:3;jenkins-hbase4:34917] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-24 20:10:43,778 DEBUG [RS:3;jenkins-hbase4:34917] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-24 20:10:43,778 DEBUG [RS:3;jenkins-hbase4:34917] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-24 20:10:43,778 INFO [RS:3;jenkins-hbase4:34917] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-24 20:10:43,778 INFO [RS:3;jenkins-hbase4:34917] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-24 20:10:43,782 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-24 20:10:43,786 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:43,787 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:43,788 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-24 20:10:43,792 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-24 20:10:43,795 DEBUG [hconnection-0xb35bd70-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-24 20:10:43,799 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48642, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-24 20:10:43,805 DEBUG [hconnection-0xb35bd70-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-24 20:10:43,808 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37860, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-24 20:10:43,812 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:43,812 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:43,822 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:39247] to rsgroup master 2023-07-24 20:10:43,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-24 20:10:43,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.CallRunner(144): callId: 20 service: MasterService methodName: ExecMasterService size: 118 connection: 172.31.14.131:36610 deadline: 1690230643821, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. 2023-07-24 20:10:43,824 WARN [Listener at localhost/44145] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.setUpBeforeMethod(TestRSGroupsBase.java:136) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.beforeMethod(TestRSGroupsBasics.java:77) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-24 20:10:43,826 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:10:43,828 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:43,828 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:43,829 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:34371, jenkins-hbase4.apache.org:34483, jenkins-hbase4.apache.org:34917, jenkins-hbase4.apache.org:39791], Tables:[hbase:meta, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-24 20:10:43,834 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-24 20:10:43,834 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-24 20:10:43,835 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBasics(260): testClearNotProcessedDeadServer 2023-07-24 20:10:43,836 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-24 20:10:43,837 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-24 20:10:43,838 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup deadServerGroup 2023-07-24 20:10:43,842 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:43,843 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:43,843 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/deadServerGroup 2023-07-24 20:10:43,847 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 6 2023-07-24 20:10:43,851 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-24 20:10:43,855 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:43,855 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:43,859 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:34371] to rsgroup deadServerGroup 2023-07-24 20:10:43,863 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:43,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:43,866 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/deadServerGroup 2023-07-24 20:10:43,866 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 6 2023-07-24 20:10:43,869 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(238): Moving server region 1588230740, which do not belong to RSGroup deadServerGroup 2023-07-24 20:10:43,870 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-24 20:10:43,870 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 20:10:43,870 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 20:10:43,870 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 20:10:43,870 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 20:10:43,873 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, REOPEN/MOVE 2023-07-24 20:10:43,873 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, REOPEN/MOVE 2023-07-24 20:10:43,874 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(286): Moving 1 region(s) to group default, current retry=0 2023-07-24 20:10:43,875 INFO [PEWorker-2] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,34371,1690229440021, state=CLOSING 2023-07-24 20:10:43,876 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-07-24 20:10:43,876 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-24 20:10:43,876 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE; CloseRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,34371,1690229440021}] 2023-07-24 20:10:43,881 INFO [RS:3;jenkins-hbase4:34917] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34917%2C1690229443609, suffix=, logDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,34917,1690229443609, archiveDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/oldWALs, maxLogs=32 2023-07-24 20:10:43,909 DEBUG [RS-EventLoopGroup-7-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK] 2023-07-24 20:10:43,910 DEBUG [RS-EventLoopGroup-7-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK] 2023-07-24 20:10:43,910 DEBUG [RS-EventLoopGroup-7-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK] 2023-07-24 20:10:43,915 INFO [RS:3;jenkins-hbase4:34917] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,34917,1690229443609/jenkins-hbase4.apache.org%2C34917%2C1690229443609.1690229443883 2023-07-24 20:10:43,918 DEBUG [RS:3;jenkins-hbase4:34917] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK], DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK], DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK]] 2023-07-24 20:10:44,038 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 1588230740 2023-07-24 20:10:44,039 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-24 20:10:44,039 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-24 20:10:44,039 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-24 20:10:44,039 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-24 20:10:44,039 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-24 20:10:44,040 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=2.49 KB heapSize=5 KB 2023-07-24 20:10:44,133 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.31 KB at sequenceid=14 (bloomFilter=false), to=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/.tmp/info/14fae0676b7b4b9dbef6a9c00b67e86e 2023-07-24 20:10:44,205 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=184 B at sequenceid=14 (bloomFilter=false), to=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/.tmp/table/a90c3b62969d4417a8b965913cf8fbbc 2023-07-24 20:10:44,216 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/.tmp/info/14fae0676b7b4b9dbef6a9c00b67e86e as hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/info/14fae0676b7b4b9dbef6a9c00b67e86e 2023-07-24 20:10:44,226 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/info/14fae0676b7b4b9dbef6a9c00b67e86e, entries=20, sequenceid=14, filesize=7.0 K 2023-07-24 20:10:44,229 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/.tmp/table/a90c3b62969d4417a8b965913cf8fbbc as hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/table/a90c3b62969d4417a8b965913cf8fbbc 2023-07-24 20:10:44,243 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/table/a90c3b62969d4417a8b965913cf8fbbc, entries=4, sequenceid=14, filesize=4.8 K 2023-07-24 20:10:44,245 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~2.49 KB/2550, heapSize ~4.72 KB/4832, currentSize=0 B/0 for 1588230740 in 205ms, sequenceid=14, compaction requested=false 2023-07-24 20:10:44,247 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-24 20:10:44,260 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/recovered.edits/17.seqid, newMaxSeqId=17, maxSeqId=1 2023-07-24 20:10:44,261 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-24 20:10:44,262 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-24 20:10:44,262 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-24 20:10:44,262 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3513): Adding 1588230740 move to jenkins-hbase4.apache.org,34917,1690229443609 record at close sequenceid=14 2023-07-24 20:10:44,265 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 1588230740 2023-07-24 20:10:44,266 WARN [PEWorker-5] zookeeper.MetaTableLocator(225): Tried to set null ServerName in hbase:meta; skipping -- ServerName required 2023-07-24 20:10:44,269 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-07-24 20:10:44,269 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; CloseRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,34371,1690229440021 in 390 msec 2023-07-24 20:10:44,270 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,34917,1690229443609; forceNewPlan=false, retain=false 2023-07-24 20:10:44,420 INFO [jenkins-hbase4:39247] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-24 20:10:44,420 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,34917,1690229443609, state=OPENING 2023-07-24 20:10:44,423 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-07-24 20:10:44,423 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-24 20:10:44,423 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=12, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,34917,1690229443609}] 2023-07-24 20:10:44,578 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:44,578 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-24 20:10:44,580 INFO [RS-EventLoopGroup-7-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51890, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-24 20:10:44,585 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-24 20:10:44,585 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-24 20:10:44,588 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34917%2C1690229443609.meta, suffix=.meta, logDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,34917,1690229443609, archiveDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/oldWALs, maxLogs=32 2023-07-24 20:10:44,605 DEBUG [RS-EventLoopGroup-7-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK] 2023-07-24 20:10:44,606 DEBUG [RS-EventLoopGroup-7-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK] 2023-07-24 20:10:44,607 DEBUG [RS-EventLoopGroup-7-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK] 2023-07-24 20:10:44,609 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,34917,1690229443609/jenkins-hbase4.apache.org%2C34917%2C1690229443609.meta.1690229444589.meta 2023-07-24 20:10:44,609 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK], DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK], DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK]] 2023-07-24 20:10:44,610 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-24 20:10:44,610 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-24 20:10:44,610 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-24 20:10:44,610 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-24 20:10:44,610 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-24 20:10:44,610 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:44,610 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-24 20:10:44,610 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-24 20:10:44,612 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-24 20:10:44,614 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/info 2023-07-24 20:10:44,614 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/info 2023-07-24 20:10:44,614 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-24 20:10:44,625 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/info/14fae0676b7b4b9dbef6a9c00b67e86e 2023-07-24 20:10:44,625 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:10:44,626 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-24 20:10:44,627 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/rep_barrier 2023-07-24 20:10:44,627 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/rep_barrier 2023-07-24 20:10:44,627 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-24 20:10:44,628 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:10:44,628 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-24 20:10:44,629 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/table 2023-07-24 20:10:44,629 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/table 2023-07-24 20:10:44,629 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-24 20:10:44,638 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/table/a90c3b62969d4417a8b965913cf8fbbc 2023-07-24 20:10:44,638 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:10:44,639 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740 2023-07-24 20:10:44,642 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740 2023-07-24 20:10:44,645 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-24 20:10:44,647 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-24 20:10:44,649 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=18; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=9954803680, jitterRate=-0.07288666069507599}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-24 20:10:44,649 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-24 20:10:44,651 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=14, masterSystemTime=1690229444577 2023-07-24 20:10:44,655 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-24 20:10:44,655 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-24 20:10:44,656 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,34917,1690229443609, state=OPEN 2023-07-24 20:10:44,658 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-07-24 20:10:44,658 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-24 20:10:44,660 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=12 2023-07-24 20:10:44,660 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=12, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,34917,1690229443609 in 235 msec 2023-07-24 20:10:44,662 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, REOPEN/MOVE in 789 msec 2023-07-24 20:10:44,874 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] procedure.ProcedureSyncWait(216): waitFor pid=12 2023-07-24 20:10:44,875 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(261): All regions from [jenkins-hbase4.apache.org,34371,1690229440021] are moved back to default 2023-07-24 20:10:44,875 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(438): Move servers done: default => deadServerGroup 2023-07-24 20:10:44,875 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-24 20:10:44,879 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:44,879 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:44,883 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=deadServerGroup 2023-07-24 20:10:44,883 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-24 20:10:44,888 DEBUG [Listener at localhost/44145] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-24 20:10:44,892 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:48652, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-24 20:10:44,892 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34371] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,34371,1690229440021' ***** 2023-07-24 20:10:44,892 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34371] regionserver.HRegionServer(2311): STOPPED: Called by admin client hconnection-0x4636fd6e 2023-07-24 20:10:44,893 INFO [RS:1;jenkins-hbase4:34371] regionserver.HRegionServer(1109): Stopping infoServer 2023-07-24 20:10:44,903 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:10:44,906 INFO [RS:1;jenkins-hbase4:34371] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.w.WebAppContext@1fca8265{regionserver,/,null,STOPPED}{file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver} 2023-07-24 20:10:44,911 INFO [RS:1;jenkins-hbase4:34371] server.AbstractConnector(383): Stopped ServerConnector@14c5b0ab{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-24 20:10:44,911 INFO [RS:1;jenkins-hbase4:34371] session.HouseKeeper(149): node0 Stopped scavenging 2023-07-24 20:10:44,912 INFO [RS:1;jenkins-hbase4:34371] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@64c0781{static,/static,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/static/,STOPPED} 2023-07-24 20:10:44,913 INFO [RS:1;jenkins-hbase4:34371] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@4cbd39c9{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/hadoop.log.dir/,STOPPED} 2023-07-24 20:10:44,916 INFO [RS:1;jenkins-hbase4:34371] regionserver.HeapMemoryManager(220): Stopping 2023-07-24 20:10:44,916 INFO [RS:1;jenkins-hbase4:34371] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-24 20:10:44,916 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-24 20:10:44,916 INFO [RS:1;jenkins-hbase4:34371] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-24 20:10:44,916 INFO [RS:1;jenkins-hbase4:34371] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,34371,1690229440021 2023-07-24 20:10:44,916 DEBUG [RS:1;jenkins-hbase4:34371] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x141e81d2 to 127.0.0.1:62881 2023-07-24 20:10:44,916 DEBUG [RS:1;jenkins-hbase4:34371] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:10:44,917 INFO [RS:1;jenkins-hbase4:34371] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,34371,1690229440021; all regions closed. 2023-07-24 20:10:44,935 DEBUG [RS:1;jenkins-hbase4:34371] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/oldWALs 2023-07-24 20:10:44,935 INFO [RS:1;jenkins-hbase4:34371] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34371%2C1690229440021.meta:.meta(num 1690229442285) 2023-07-24 20:10:44,946 DEBUG [RS:1;jenkins-hbase4:34371] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/oldWALs 2023-07-24 20:10:44,946 INFO [RS:1;jenkins-hbase4:34371] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34371%2C1690229440021:(num 1690229442118) 2023-07-24 20:10:44,946 DEBUG [RS:1;jenkins-hbase4:34371] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:10:44,947 INFO [RS:1;jenkins-hbase4:34371] regionserver.LeaseManager(133): Closed leases 2023-07-24 20:10:44,947 INFO [RS:1;jenkins-hbase4:34371] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-24 20:10:44,947 INFO [RS:1;jenkins-hbase4:34371] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-24 20:10:44,947 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-24 20:10:44,948 INFO [RS:1;jenkins-hbase4:34371] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-24 20:10:44,948 INFO [RS:1;jenkins-hbase4:34371] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-24 20:10:44,949 INFO [RS:1;jenkins-hbase4:34371] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:34371 2023-07-24 20:10:44,955 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-24 20:10:44,963 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34371,1690229440021 2023-07-24 20:10:44,963 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:10:44,963 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34917-0x10198f4f3ce000b, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34371,1690229440021 2023-07-24 20:10:44,963 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:10:44,963 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34371-0x10198f4f3ce0002, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34371,1690229440021 2023-07-24 20:10:44,963 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34371-0x10198f4f3ce0002, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:10:44,963 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34371,1690229440021 2023-07-24 20:10:44,964 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:10:44,963 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34917-0x10198f4f3ce000b, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:10:44,966 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,34371,1690229440021] 2023-07-24 20:10:44,966 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,34371,1690229440021; numProcessing=1 2023-07-24 20:10:44,967 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:44,967 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:44,967 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34917-0x10198f4f3ce000b, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:44,968 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:44,968 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:44,968 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,34371,1690229440021 already deleted, retry=false 2023-07-24 20:10:44,968 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34917-0x10198f4f3ce000b, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:44,968 INFO [RegionServerTracker-0] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,34371,1690229440021 on jenkins-hbase4.apache.org,39247,1690229437650 2023-07-24 20:10:44,970 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:44,970 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34917-0x10198f4f3ce000b, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:44,970 INFO [zk-event-processor-pool-0] replication.ReplicationTrackerZKImpl$OtherRegionServerWatcher(124): /hbase/rs/jenkins-hbase4.apache.org,34371,1690229440021 znode expired, triggering replicatorRemoved event 2023-07-24 20:10:44,971 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:44,971 INFO [zk-event-processor-pool-0] replication.ReplicationTrackerZKImpl$OtherRegionServerWatcher(124): /hbase/rs/jenkins-hbase4.apache.org,34371,1690229440021 znode expired, triggering replicatorRemoved event 2023-07-24 20:10:44,971 INFO [zk-event-processor-pool-0] replication.ReplicationTrackerZKImpl$OtherRegionServerWatcher(124): /hbase/rs/jenkins-hbase4.apache.org,34371,1690229440021 znode expired, triggering replicatorRemoved event 2023-07-24 20:10:44,974 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:44,975 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:44,975 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34917-0x10198f4f3ce000b, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:44,975 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:44,975 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:44,976 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:44,976 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34917-0x10198f4f3ce000b, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:44,976 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:44,977 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34917-0x10198f4f3ce000b, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:44,981 DEBUG [RegionServerTracker-0] procedure2.ProcedureExecutor(1029): Stored pid=15, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,34371,1690229440021, splitWal=true, meta=false 2023-07-24 20:10:44,981 INFO [RegionServerTracker-0] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=15 for jenkins-hbase4.apache.org,34371,1690229440021 (carryingMeta=false) jenkins-hbase4.apache.org,34371,1690229440021/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@3f153c60[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-24 20:10:44,982 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,39247,1690229437650] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(787): Updating default servers. 2023-07-24 20:10:44,985 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=15, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,34371,1690229440021, splitWal=true, meta=false 2023-07-24 20:10:44,986 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,39247,1690229437650] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:44,987 INFO [PEWorker-1] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,34371,1690229440021 had 0 regions 2023-07-24 20:10:44,987 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,39247,1690229437650] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:44,988 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,39247,1690229437650] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/deadServerGroup 2023-07-24 20:10:44,990 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=15, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,34371,1690229440021, splitWal=true, meta=false, isMeta: false 2023-07-24 20:10:44,990 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,39247,1690229437650] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 6 2023-07-24 20:10:44,994 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,34371,1690229440021-splitting 2023-07-24 20:10:44,996 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,34371,1690229440021-splitting dir is empty, no logs to split. 2023-07-24 20:10:44,996 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,34371,1690229440021 WAL count=0, meta=false 2023-07-24 20:10:44,999 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,39247,1690229437650] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(792): Updated with servers: 3 2023-07-24 20:10:45,002 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,34371,1690229440021-splitting dir is empty, no logs to split. 2023-07-24 20:10:45,002 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,34371,1690229440021 WAL count=0, meta=false 2023-07-24 20:10:45,002 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,34371,1690229440021 WAL splitting is done? wals=0, meta=false 2023-07-24 20:10:45,005 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=deadServerGroup 2023-07-24 20:10:45,005 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-24 20:10:45,008 INFO [PEWorker-1] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,34371,1690229440021 failed, ignore...File hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,34371,1690229440021-splitting does not exist. 2023-07-24 20:10:45,011 INFO [PEWorker-1] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,34371,1690229440021 after splitting done 2023-07-24 20:10:45,012 DEBUG [PEWorker-1] master.DeadServer(114): Removed jenkins-hbase4.apache.org,34371,1690229440021 from processing; numProcessing=0 2023-07-24 20:10:45,012 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:45,012 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:45,015 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,34371,1690229440021, splitWal=true, meta=false in 40 msec 2023-07-24 20:10:45,020 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-24 20:10:45,020 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-24 20:10:45,020 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-24 20:10:45,022 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-24 20:10:45,022 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-24 20:10:45,028 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-24 20:10:45,034 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:45,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/deadServerGroup 2023-07-24 20:10:45,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 5 2023-07-24 20:10:45,040 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-24 20:10:45,042 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-24 20:10:45,042 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-24 20:10:45,042 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-24 20:10:45,043 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:34371] to rsgroup default 2023-07-24 20:10:45,043 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(258): Dropping jenkins-hbase4.apache.org:34371 during move-to-default rsgroup because not online 2023-07-24 20:10:45,046 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:45,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/deadServerGroup 2023-07-24 20:10:45,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-24 20:10:45,050 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(286): Moving 0 region(s) to group deadServerGroup, current retry=0 2023-07-24 20:10:45,050 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(261): All regions from [] are moved back to deadServerGroup 2023-07-24 20:10:45,050 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(438): Move servers done: deadServerGroup => default 2023-07-24 20:10:45,050 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-24 20:10:45,052 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup deadServerGroup 2023-07-24 20:10:45,057 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:45,057 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-24 20:10:45,061 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-24 20:10:45,066 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase(152): Restoring servers: 1 2023-07-24 20:10:45,079 INFO [Listener at localhost/44145] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-24 20:10:45,079 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 20:10:45,079 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-24 20:10:45,079 INFO [Listener at localhost/44145] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-24 20:10:45,079 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 20:10:45,079 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-24 20:10:45,080 INFO [Listener at localhost/44145] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-24 20:10:45,081 INFO [Listener at localhost/44145] ipc.NettyRpcServer(120): Bind to /172.31.14.131:45209 2023-07-24 20:10:45,082 INFO [Listener at localhost/44145] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-24 20:10:45,083 DEBUG [Listener at localhost/44145] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-24 20:10:45,084 INFO [Listener at localhost/44145] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:10:45,085 INFO [Listener at localhost/44145] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:10:45,087 INFO [Listener at localhost/44145] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:45209 connecting to ZooKeeper ensemble=127.0.0.1:62881 2023-07-24 20:10:45,091 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:452090x0, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-24 20:10:45,093 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:45209-0x10198f4f3ce000d connected 2023-07-24 20:10:45,093 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(162): regionserver:45209-0x10198f4f3ce000d, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-24 20:10:45,094 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(162): regionserver:45209-0x10198f4f3ce000d, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/running 2023-07-24 20:10:45,095 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): regionserver:45209-0x10198f4f3ce000d, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-24 20:10:45,096 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=45209 2023-07-24 20:10:45,099 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=45209 2023-07-24 20:10:45,102 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=45209 2023-07-24 20:10:45,104 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=45209 2023-07-24 20:10:45,105 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=45209 2023-07-24 20:10:45,107 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2023-07-24 20:10:45,107 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter) 2023-07-24 20:10:45,108 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'securityheaders' (class=org.apache.hadoop.hbase.http.SecurityHeadersFilter) 2023-07-24 20:10:45,108 INFO [Listener at localhost/44145] http.HttpServer(879): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context regionserver 2023-07-24 20:10:45,108 INFO [Listener at localhost/44145] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2023-07-24 20:10:45,108 INFO [Listener at localhost/44145] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2023-07-24 20:10:45,109 INFO [Listener at localhost/44145] http.HttpServer(783): ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint. 2023-07-24 20:10:45,109 INFO [Listener at localhost/44145] http.HttpServer(1146): Jetty bound to port 39695 2023-07-24 20:10:45,109 INFO [Listener at localhost/44145] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-24 20:10:45,114 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:10:45,114 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@64470437{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/hadoop.log.dir/,AVAILABLE} 2023-07-24 20:10:45,115 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:10:45,115 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@3bcb01cf{static,/static,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/static/,AVAILABLE} 2023-07-24 20:10:45,119 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34371-0x10198f4f3ce0002, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 20:10:45,120 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34371-0x10198f4f3ce0002, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 20:10:45,120 INFO [RS:1;jenkins-hbase4:34371] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,34371,1690229440021; zookeeper connection closed. 2023-07-24 20:10:45,120 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@70875e02] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@70875e02 2023-07-24 20:10:45,127 INFO [Listener at localhost/44145] webapp.StandardDescriptorProcessor(277): NO JSP Support for /, did not find org.apache.hbase.thirdparty.org.eclipse.jetty.jsp.JettyJspServlet 2023-07-24 20:10:45,128 INFO [Listener at localhost/44145] session.DefaultSessionIdManager(334): DefaultSessionIdManager workerName=node0 2023-07-24 20:10:45,128 INFO [Listener at localhost/44145] session.DefaultSessionIdManager(339): No SessionScavenger set, using defaults 2023-07-24 20:10:45,129 INFO [Listener at localhost/44145] session.HouseKeeper(132): node0 Scavenging every 660000ms 2023-07-24 20:10:45,130 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:10:45,131 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.w.WebAppContext@25fb09ce{regionserver,/,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver/,AVAILABLE}{file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver} 2023-07-24 20:10:45,133 INFO [Listener at localhost/44145] server.AbstractConnector(333): Started ServerConnector@7790aa8{HTTP/1.1, (http/1.1)}{0.0.0.0:39695} 2023-07-24 20:10:45,133 INFO [Listener at localhost/44145] server.Server(415): Started @13700ms 2023-07-24 20:10:45,137 INFO [RS:4;jenkins-hbase4:45209] regionserver.HRegionServer(951): ClusterId : dc54b028-3544-4133-8f0e-d550a19bc790 2023-07-24 20:10:45,138 DEBUG [RS:4;jenkins-hbase4:45209] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-24 20:10:45,141 DEBUG [RS:4;jenkins-hbase4:45209] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-24 20:10:45,141 DEBUG [RS:4;jenkins-hbase4:45209] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-24 20:10:45,143 DEBUG [RS:4;jenkins-hbase4:45209] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-24 20:10:45,145 DEBUG [RS:4;jenkins-hbase4:45209] zookeeper.ReadOnlyZKClient(139): Connect 0x44764448 to 127.0.0.1:62881 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-24 20:10:45,150 DEBUG [RS:4;jenkins-hbase4:45209] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4ab1a2b6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-24 20:10:45,150 DEBUG [RS:4;jenkins-hbase4:45209] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4e8e2f38, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-24 20:10:45,159 DEBUG [RS:4;jenkins-hbase4:45209] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:4;jenkins-hbase4:45209 2023-07-24 20:10:45,159 INFO [RS:4;jenkins-hbase4:45209] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-24 20:10:45,159 INFO [RS:4;jenkins-hbase4:45209] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-24 20:10:45,159 DEBUG [RS:4;jenkins-hbase4:45209] regionserver.HRegionServer(1022): About to register with Master. 2023-07-24 20:10:45,160 INFO [RS:4;jenkins-hbase4:45209] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,39247,1690229437650 with isa=jenkins-hbase4.apache.org/172.31.14.131:45209, startcode=1690229445078 2023-07-24 20:10:45,160 DEBUG [RS:4;jenkins-hbase4:45209] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-24 20:10:45,163 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55797, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.4 (auth:SIMPLE), service=RegionServerStatusService 2023-07-24 20:10:45,163 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=39247] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:45,163 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,39247,1690229437650] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(787): Updating default servers. 2023-07-24 20:10:45,164 DEBUG [RS:4;jenkins-hbase4:45209] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4 2023-07-24 20:10:45,164 DEBUG [RS:4;jenkins-hbase4:45209] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:37609 2023-07-24 20:10:45,164 DEBUG [RS:4;jenkins-hbase4:45209] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=36023 2023-07-24 20:10:45,166 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:10:45,166 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:10:45,166 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:10:45,166 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34917-0x10198f4f3ce000b, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:10:45,167 DEBUG [RS:4;jenkins-hbase4:45209] zookeeper.ZKUtil(162): regionserver:45209-0x10198f4f3ce000d, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:45,167 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,45209,1690229445078] 2023-07-24 20:10:45,167 WARN [RS:4;jenkins-hbase4:45209] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-24 20:10:45,167 INFO [RS:4;jenkins-hbase4:45209] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-24 20:10:45,167 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:45,167 DEBUG [RS:4;jenkins-hbase4:45209] regionserver.HRegionServer(1948): logDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:45,167 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:45,167 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,39247,1690229437650] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:45,168 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:45,168 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34917-0x10198f4f3ce000b, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:45,168 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:45,168 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,39247,1690229437650] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 2 2023-07-24 20:10:45,168 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:45,169 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34917-0x10198f4f3ce000b, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:45,169 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:45,176 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:45,176 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:45,176 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,39247,1690229437650] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(792): Updated with servers: 4 2023-07-24 20:10:45,176 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34917-0x10198f4f3ce000b, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:45,177 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34917-0x10198f4f3ce000b, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:45,177 DEBUG [RS:4;jenkins-hbase4:45209] zookeeper.ZKUtil(162): regionserver:45209-0x10198f4f3ce000d, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:45,177 DEBUG [RS:4;jenkins-hbase4:45209] zookeeper.ZKUtil(162): regionserver:45209-0x10198f4f3ce000d, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:45,177 DEBUG [RS:4;jenkins-hbase4:45209] zookeeper.ZKUtil(162): regionserver:45209-0x10198f4f3ce000d, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:45,178 DEBUG [RS:4;jenkins-hbase4:45209] zookeeper.ZKUtil(162): regionserver:45209-0x10198f4f3ce000d, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:45,179 DEBUG [RS:4;jenkins-hbase4:45209] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-24 20:10:45,179 INFO [RS:4;jenkins-hbase4:45209] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-24 20:10:45,181 INFO [RS:4;jenkins-hbase4:45209] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-24 20:10:45,181 INFO [RS:4;jenkins-hbase4:45209] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-24 20:10:45,181 INFO [RS:4;jenkins-hbase4:45209] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:45,181 INFO [RS:4;jenkins-hbase4:45209] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-24 20:10:45,184 INFO [RS:4;jenkins-hbase4:45209] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:45,184 DEBUG [RS:4;jenkins-hbase4:45209] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:45,184 DEBUG [RS:4;jenkins-hbase4:45209] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:45,184 DEBUG [RS:4;jenkins-hbase4:45209] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:45,184 DEBUG [RS:4;jenkins-hbase4:45209] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:45,184 DEBUG [RS:4;jenkins-hbase4:45209] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:45,184 DEBUG [RS:4;jenkins-hbase4:45209] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-24 20:10:45,184 DEBUG [RS:4;jenkins-hbase4:45209] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:45,184 DEBUG [RS:4;jenkins-hbase4:45209] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:45,185 DEBUG [RS:4;jenkins-hbase4:45209] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:45,185 DEBUG [RS:4;jenkins-hbase4:45209] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:10:45,188 INFO [RS:4;jenkins-hbase4:45209] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:45,188 INFO [RS:4;jenkins-hbase4:45209] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:45,188 INFO [RS:4;jenkins-hbase4:45209] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:45,199 INFO [RS:4;jenkins-hbase4:45209] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-24 20:10:45,199 INFO [RS:4;jenkins-hbase4:45209] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45209,1690229445078-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:10:45,212 INFO [RS:4;jenkins-hbase4:45209] regionserver.Replication(203): jenkins-hbase4.apache.org,45209,1690229445078 started 2023-07-24 20:10:45,212 INFO [RS:4;jenkins-hbase4:45209] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,45209,1690229445078, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:45209, sessionid=0x10198f4f3ce000d 2023-07-24 20:10:45,212 DEBUG [RS:4;jenkins-hbase4:45209] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-24 20:10:45,212 DEBUG [RS:4;jenkins-hbase4:45209] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:45,212 DEBUG [RS:4;jenkins-hbase4:45209] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,45209,1690229445078' 2023-07-24 20:10:45,212 DEBUG [RS:4;jenkins-hbase4:45209] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 20:10:45,213 DEBUG [RS:4;jenkins-hbase4:45209] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 20:10:45,213 DEBUG [RS:4;jenkins-hbase4:45209] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-24 20:10:45,214 DEBUG [RS:4;jenkins-hbase4:45209] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-24 20:10:45,214 DEBUG [RS:4;jenkins-hbase4:45209] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:45,214 DEBUG [RS:4;jenkins-hbase4:45209] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,45209,1690229445078' 2023-07-24 20:10:45,214 DEBUG [RS:4;jenkins-hbase4:45209] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-24 20:10:45,214 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-24 20:10:45,214 DEBUG [RS:4;jenkins-hbase4:45209] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-24 20:10:45,214 DEBUG [RS:4;jenkins-hbase4:45209] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-24 20:10:45,215 INFO [RS:4;jenkins-hbase4:45209] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-24 20:10:45,215 INFO [RS:4;jenkins-hbase4:45209] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-24 20:10:45,218 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:45,218 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:45,220 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-24 20:10:45,221 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-24 20:10:45,227 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:45,227 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:45,230 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:39247] to rsgroup master 2023-07-24 20:10:45,230 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-24 20:10:45,230 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.CallRunner(144): callId: 69 service: MasterService methodName: ExecMasterService size: 118 connection: 172.31.14.131:36610 deadline: 1690230645230, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. 2023-07-24 20:10:45,231 WARN [Listener at localhost/44145] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.afterMethod(TestRSGroupsBasics.java:82) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-24 20:10:45,232 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:10:45,233 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:45,233 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:45,234 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:34483, jenkins-hbase4.apache.org:34917, jenkins-hbase4.apache.org:39791, jenkins-hbase4.apache.org:45209], Tables:[hbase:meta, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-24 20:10:45,235 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-24 20:10:45,235 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-24 20:10:45,261 INFO [Listener at localhost/44145] hbase.ResourceChecker(175): after: rsgroup.TestRSGroupsBasics#testClearNotProcessedDeadServer Thread=480 (was 422) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45209 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: LeaseRenewer:jenkins.hfs.3@localhost:37609 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_743653681_17 at /127.0.0.1:54074 [Receiving block BP-1903532105-172.31.14.131-1690229433619:blk_1073741843_1019] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1892626770-640 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:4;jenkins-hbase4:45209-longCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) org.apache.hadoop.hbase.util.StealJobQueue.take(StealJobQueue.java:101) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1540241166_17 at /127.0.0.1:42776 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (539191006) connection to localhost/127.0.0.1:37609 from jenkins.hfs.3 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: hconnection-0xb35bd70-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:3;jenkins-hbase4:34917-longCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) org.apache.hadoop.hbase.util.StealJobQueue.take(StealJobQueue.java:101) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1892626770-638 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:4;jenkins-hbase4:45209 java.lang.Object.wait(Native Method) org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:81) org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:64) org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1092) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:175) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:123) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:159) java.security.AccessController.doPrivileged(Native Method) javax.security.auth.Subject.doAs(Subject.java:360) org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1873) org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:319) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:156) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2045478459_17 at /127.0.0.1:54072 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait0(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:182) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:302) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:366) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1751520955-748 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xb35bd70-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-1903532105-172.31.14.131-1690229433619:blk_1073741840_1016, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_743653681_17 at /127.0.0.1:54030 [Receiving block BP-1903532105-172.31.14.131-1690229433619:blk_1073741840_1016] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1892626770-642 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-1903532105-172.31.14.131-1690229433619:blk_1073741840_1016, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1751520955-745 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.metaPriority.FPBQ.Fifo.handler=0,queue=0,port=45209 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: hconnection-0xb35bd70-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait0(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:182) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:302) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:366) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=45209 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34917 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_743653681_17 at /127.0.0.1:44880 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.metaPriority.FPBQ.Fifo.handler=0,queue=0,port=34917 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: qtp1751520955-743 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62881@0x01c5fe6f sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$92/589794509.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=34917 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62881@0x44764448-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: qtp1892626770-637-acceptor-0@65e45b43-ServerConnector@41896d02{HTTP/1.1, (http/1.1)}{0.0.0.0:37561} sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:421) sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:249) org.apache.hbase.thirdparty.org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:388) org.apache.hbase.thirdparty.org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:704) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=45209 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: hconnection-0x229abe1d-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1751520955-746 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xb35bd70-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=45209 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=0,queue=0,port=34917 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RS:3;jenkins-hbase4:34917 java.lang.Object.wait(Native Method) org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:81) org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:64) org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1092) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:175) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:123) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:159) java.security.AccessController.doPrivileged(Native Method) javax.security.auth.Subject.doAs(Subject.java:360) org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1873) org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:319) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:156) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62881@0x44764448 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$92/589794509.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait0(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:182) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:302) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:366) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1892626770-636 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$$Lambda$75/1893131221.run(Unknown Source) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1751520955-741 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$$Lambda$75/1893131221.run(Unknown Source) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45209 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: jenkins-hbase4:34917Replication Statistics #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34917 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62881@0x44764448-SendThread(127.0.0.1:62881) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_743653681_17 at /127.0.0.1:42942 [Receiving block BP-1903532105-172.31.14.131-1690229433619:blk_1073741843_1019] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1892626770-641 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_743653681_17 at /127.0.0.1:44884 [Receiving block BP-1903532105-172.31.14.131-1690229433619:blk_1073741840_1016] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1751520955-744 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xb35bd70-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34917 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_743653681_17 at /127.0.0.1:44906 [Receiving block BP-1903532105-172.31.14.131-1690229433619:blk_1073741843_1019] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62881@0x01c5fe6f-SendThread(127.0.0.1:62881) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: qtp1892626770-643 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-1903532105-172.31.14.131-1690229433619:blk_1073741843_1019, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=34917 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: AsyncFSWAL-0-hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4-prefix:jenkins-hbase4.apache.org,34917,1690229443609.meta sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1892626770-639 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-7-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait0(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:182) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:302) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:366) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62881@0x01c5fe6f-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (539191006) connection to localhost/127.0.0.1:37609 from jenkins.hfs.4 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: jenkins-hbase4:45209Replication Statistics #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=0,queue=0,port=45209 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1540241166_17 at /127.0.0.1:44902 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-1903532105-172.31.14.131-1690229433619:blk_1073741840_1016, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Session-HouseKeeper-321b1c49-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34917 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: PacketResponder: BP-1903532105-172.31.14.131-1690229433619:blk_1073741843_1019, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45209 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: Session-HouseKeeper-f04bfb2-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_743653681_17 at /127.0.0.1:42908 [Receiving block BP-1903532105-172.31.14.131-1690229433619:blk_1073741840_1016] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-1903532105-172.31.14.131-1690229433619:blk_1073741843_1019, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1751520955-747 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: AsyncFSWAL-0-hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4-prefix:jenkins-hbase4.apache.org,34917,1690229443609 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=45209 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=34917 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=45209 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: qtp1751520955-742-acceptor-0@75229ae5-ServerConnector@7790aa8{HTTP/1.1, (http/1.1)}{0.0.0.0:39695} sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:421) sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:249) org.apache.hbase.thirdparty.org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:388) org.apache.hbase.thirdparty.org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:704) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=34917 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) - Thread LEAK? -, OpenFileDescriptor=723 (was 673) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=394 (was 393) - SystemLoadAverage LEAK? -, ProcessCount=176 (was 176), AvailableMemoryMB=5968 (was 6070) 2023-07-24 20:10:45,278 INFO [Listener at localhost/44145] hbase.ResourceChecker(147): before: rsgroup.TestRSGroupsBasics#testDefaultNamespaceCreateAndAssign Thread=480, OpenFileDescriptor=723, MaxFileDescriptor=60000, SystemLoadAverage=394, ProcessCount=176, AvailableMemoryMB=5967 2023-07-24 20:10:45,279 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase(132): testDefaultNamespaceCreateAndAssign 2023-07-24 20:10:45,284 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:45,285 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:45,286 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-24 20:10:45,286 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-24 20:10:45,286 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-24 20:10:45,288 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-24 20:10:45,288 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-24 20:10:45,289 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-24 20:10:45,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:45,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-24 20:10:45,297 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-24 20:10:45,304 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase(152): Restoring servers: 0 2023-07-24 20:10:45,305 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-24 20:10:45,308 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:45,309 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:45,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-24 20:10:45,315 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-24 20:10:45,318 INFO [RS:4;jenkins-hbase4:45209] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45209%2C1690229445078, suffix=, logDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,45209,1690229445078, archiveDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/oldWALs, maxLogs=32 2023-07-24 20:10:45,319 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:45,320 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:45,322 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:39247] to rsgroup master 2023-07-24 20:10:45,323 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-24 20:10:45,323 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.CallRunner(144): callId: 97 service: MasterService methodName: ExecMasterService size: 118 connection: 172.31.14.131:36610 deadline: 1690230645322, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. 2023-07-24 20:10:45,323 WARN [Listener at localhost/44145] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.setUpBeforeMethod(TestRSGroupsBase.java:136) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.beforeMethod(TestRSGroupsBasics.java:77) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-24 20:10:45,325 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:10:45,326 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:45,326 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:45,327 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:34483, jenkins-hbase4.apache.org:34917, jenkins-hbase4.apache.org:39791, jenkins-hbase4.apache.org:45209], Tables:[hbase:meta, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-24 20:10:45,328 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-24 20:10:45,328 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-24 20:10:45,329 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBasics(180): testDefaultNamespaceCreateAndAssign 2023-07-24 20:10:45,337 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.HMaster$16(3053): Client=jenkins//172.31.14.131 modify {NAME => 'default', hbase.rsgroup.name => 'default'} 2023-07-24 20:10:45,351 DEBUG [RS-EventLoopGroup-8-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK] 2023-07-24 20:10:45,351 DEBUG [RS-EventLoopGroup-8-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK] 2023-07-24 20:10:45,352 DEBUG [RS-EventLoopGroup-8-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK] 2023-07-24 20:10:45,354 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] procedure2.ProcedureExecutor(1029): Stored pid=16, state=RUNNABLE:MODIFY_NAMESPACE_PREPARE; ModifyNamespaceProcedure, namespace=default 2023-07-24 20:10:45,357 INFO [RS:4;jenkins-hbase4:45209] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,45209,1690229445078/jenkins-hbase4.apache.org%2C45209%2C1690229445078.1690229445318 2023-07-24 20:10:45,358 DEBUG [RS:4;jenkins-hbase4:45209] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK], DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK], DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK]] 2023-07-24 20:10:45,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=16 2023-07-24 20:10:45,373 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-07-24 20:10:45,376 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=16, state=SUCCESS; ModifyNamespaceProcedure, namespace=default in 36 msec 2023-07-24 20:10:45,476 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=16 2023-07-24 20:10:45,487 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'Group_testCreateAndAssign', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-24 20:10:45,489 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] procedure2.ProcedureExecutor(1029): Stored pid=17, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=Group_testCreateAndAssign 2023-07-24 20:10:45,491 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=17, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=Group_testCreateAndAssign execute state=CREATE_TABLE_PRE_OPERATION 2023-07-24 20:10:45,496 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "Group_testCreateAndAssign" procId is: 17 2023-07-24 20:10:45,497 DEBUG [PEWorker-2] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:45,497 DEBUG [PEWorker-2] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:45,498 DEBUG [PEWorker-2] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-24 20:10:45,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=17 2023-07-24 20:10:45,500 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=17, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=Group_testCreateAndAssign execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-24 20:10:45,502 WARN [RS-EventLoopGroup-5-2] ipc.NettyRpcConnection$2(294): Exception encountered while connecting to the server jenkins-hbase4.apache.org/172.31.14.131:34371 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:34371 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:155) at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:128) at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-07-24 20:10:45,504 DEBUG [RS-EventLoopGroup-5-2] ipc.FailedServers(52): Added failed server with address jenkins-hbase4.apache.org/172.31.14.131:34371 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:34371 2023-07-24 20:10:45,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=17 2023-07-24 20:10:45,605 DEBUG [PEWorker-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-24 20:10:45,607 INFO [RS-EventLoopGroup-7-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51906, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-24 20:10:45,619 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateAndAssign/cc526628eb03a103d976e6197331a7ce 2023-07-24 20:10:45,620 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateAndAssign/cc526628eb03a103d976e6197331a7ce empty. 2023-07-24 20:10:45,621 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateAndAssign/cc526628eb03a103d976e6197331a7ce 2023-07-24 20:10:45,621 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived Group_testCreateAndAssign regions 2023-07-24 20:10:45,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=17 2023-07-24 20:10:46,059 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateAndAssign/.tabledesc/.tableinfo.0000000001 2023-07-24 20:10:46,061 INFO [RegionOpenAndInit-Group_testCreateAndAssign-pool-0] regionserver.HRegion(7675): creating {ENCODED => cc526628eb03a103d976e6197331a7ce, NAME => 'Group_testCreateAndAssign,,1690229445484.cc526628eb03a103d976e6197331a7ce.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='Group_testCreateAndAssign', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp 2023-07-24 20:10:46,078 DEBUG [RegionOpenAndInit-Group_testCreateAndAssign-pool-0] regionserver.HRegion(866): Instantiated Group_testCreateAndAssign,,1690229445484.cc526628eb03a103d976e6197331a7ce.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:46,078 DEBUG [RegionOpenAndInit-Group_testCreateAndAssign-pool-0] regionserver.HRegion(1604): Closing cc526628eb03a103d976e6197331a7ce, disabling compactions & flushes 2023-07-24 20:10:46,078 INFO [RegionOpenAndInit-Group_testCreateAndAssign-pool-0] regionserver.HRegion(1626): Closing region Group_testCreateAndAssign,,1690229445484.cc526628eb03a103d976e6197331a7ce. 2023-07-24 20:10:46,078 DEBUG [RegionOpenAndInit-Group_testCreateAndAssign-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateAndAssign,,1690229445484.cc526628eb03a103d976e6197331a7ce. 2023-07-24 20:10:46,078 DEBUG [RegionOpenAndInit-Group_testCreateAndAssign-pool-0] regionserver.HRegion(1714): Acquired close lock on Group_testCreateAndAssign,,1690229445484.cc526628eb03a103d976e6197331a7ce. after waiting 0 ms 2023-07-24 20:10:46,078 DEBUG [RegionOpenAndInit-Group_testCreateAndAssign-pool-0] regionserver.HRegion(1724): Updates disabled for region Group_testCreateAndAssign,,1690229445484.cc526628eb03a103d976e6197331a7ce. 2023-07-24 20:10:46,078 INFO [RegionOpenAndInit-Group_testCreateAndAssign-pool-0] regionserver.HRegion(1838): Closed Group_testCreateAndAssign,,1690229445484.cc526628eb03a103d976e6197331a7ce. 2023-07-24 20:10:46,078 DEBUG [RegionOpenAndInit-Group_testCreateAndAssign-pool-0] regionserver.HRegion(1558): Region close journal for cc526628eb03a103d976e6197331a7ce: 2023-07-24 20:10:46,084 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=17, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=Group_testCreateAndAssign execute state=CREATE_TABLE_ADD_TO_META 2023-07-24 20:10:46,085 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"Group_testCreateAndAssign,,1690229445484.cc526628eb03a103d976e6197331a7ce.","families":{"info":[{"qualifier":"regioninfo","vlen":59,"tag":[],"timestamp":"1690229446085"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229446085"}]},"ts":"1690229446085"} 2023-07-24 20:10:46,087 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-24 20:10:46,089 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=17, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=Group_testCreateAndAssign execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-24 20:10:46,089 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCreateAndAssign","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690229446089"}]},"ts":"1690229446089"} 2023-07-24 20:10:46,090 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCreateAndAssign, state=ENABLING in hbase:meta 2023-07-24 20:10:46,093 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-24 20:10:46,094 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 20:10:46,094 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 20:10:46,094 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 20:10:46,094 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-24 20:10:46,094 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 20:10:46,094 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateAndAssign, region=cc526628eb03a103d976e6197331a7ce, ASSIGN}] 2023-07-24 20:10:46,096 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=18, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateAndAssign, region=cc526628eb03a103d976e6197331a7ce, ASSIGN 2023-07-24 20:10:46,097 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=18, ppid=17, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=Group_testCreateAndAssign, region=cc526628eb03a103d976e6197331a7ce, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45209,1690229445078; forceNewPlan=false, retain=false 2023-07-24 20:10:46,106 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=17 2023-07-24 20:10:46,248 INFO [jenkins-hbase4:39247] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-24 20:10:46,249 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=18 updating hbase:meta row=cc526628eb03a103d976e6197331a7ce, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:46,249 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateAndAssign,,1690229445484.cc526628eb03a103d976e6197331a7ce.","families":{"info":[{"qualifier":"regioninfo","vlen":59,"tag":[],"timestamp":"1690229446249"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229446249"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229446249"}]},"ts":"1690229446249"} 2023-07-24 20:10:46,252 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=19, ppid=18, state=RUNNABLE; OpenRegionProcedure cc526628eb03a103d976e6197331a7ce, server=jenkins-hbase4.apache.org,45209,1690229445078}] 2023-07-24 20:10:46,406 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:46,406 DEBUG [RSProcedureDispatcher-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-24 20:10:46,408 INFO [RS-EventLoopGroup-8-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60668, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-24 20:10:46,413 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open Group_testCreateAndAssign,,1690229445484.cc526628eb03a103d976e6197331a7ce. 2023-07-24 20:10:46,413 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => cc526628eb03a103d976e6197331a7ce, NAME => 'Group_testCreateAndAssign,,1690229445484.cc526628eb03a103d976e6197331a7ce.', STARTKEY => '', ENDKEY => ''} 2023-07-24 20:10:46,414 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table Group_testCreateAndAssign cc526628eb03a103d976e6197331a7ce 2023-07-24 20:10:46,414 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated Group_testCreateAndAssign,,1690229445484.cc526628eb03a103d976e6197331a7ce.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:46,414 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for cc526628eb03a103d976e6197331a7ce 2023-07-24 20:10:46,414 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for cc526628eb03a103d976e6197331a7ce 2023-07-24 20:10:46,415 INFO [StoreOpener-cc526628eb03a103d976e6197331a7ce-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f of region cc526628eb03a103d976e6197331a7ce 2023-07-24 20:10:46,417 DEBUG [StoreOpener-cc526628eb03a103d976e6197331a7ce-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateAndAssign/cc526628eb03a103d976e6197331a7ce/f 2023-07-24 20:10:46,418 DEBUG [StoreOpener-cc526628eb03a103d976e6197331a7ce-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateAndAssign/cc526628eb03a103d976e6197331a7ce/f 2023-07-24 20:10:46,418 INFO [StoreOpener-cc526628eb03a103d976e6197331a7ce-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cc526628eb03a103d976e6197331a7ce columnFamilyName f 2023-07-24 20:10:46,419 INFO [StoreOpener-cc526628eb03a103d976e6197331a7ce-1] regionserver.HStore(310): Store=cc526628eb03a103d976e6197331a7ce/f, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:10:46,420 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateAndAssign/cc526628eb03a103d976e6197331a7ce 2023-07-24 20:10:46,421 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateAndAssign/cc526628eb03a103d976e6197331a7ce 2023-07-24 20:10:46,425 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for cc526628eb03a103d976e6197331a7ce 2023-07-24 20:10:46,431 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateAndAssign/cc526628eb03a103d976e6197331a7ce/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 20:10:46,431 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened cc526628eb03a103d976e6197331a7ce; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=12008626080, jitterRate=0.11839045584201813}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 20:10:46,431 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for cc526628eb03a103d976e6197331a7ce: 2023-07-24 20:10:46,433 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for Group_testCreateAndAssign,,1690229445484.cc526628eb03a103d976e6197331a7ce., pid=19, masterSystemTime=1690229446406 2023-07-24 20:10:46,437 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for Group_testCreateAndAssign,,1690229445484.cc526628eb03a103d976e6197331a7ce. 2023-07-24 20:10:46,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened Group_testCreateAndAssign,,1690229445484.cc526628eb03a103d976e6197331a7ce. 2023-07-24 20:10:46,438 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=18 updating hbase:meta row=cc526628eb03a103d976e6197331a7ce, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:46,439 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"Group_testCreateAndAssign,,1690229445484.cc526628eb03a103d976e6197331a7ce.","families":{"info":[{"qualifier":"regioninfo","vlen":59,"tag":[],"timestamp":"1690229446438"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690229446438"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690229446438"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690229446438"}]},"ts":"1690229446438"} 2023-07-24 20:10:46,444 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=19, resume processing ppid=18 2023-07-24 20:10:46,444 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=19, ppid=18, state=SUCCESS; OpenRegionProcedure cc526628eb03a103d976e6197331a7ce, server=jenkins-hbase4.apache.org,45209,1690229445078 in 189 msec 2023-07-24 20:10:46,447 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=17 2023-07-24 20:10:46,449 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=17, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateAndAssign, region=cc526628eb03a103d976e6197331a7ce, ASSIGN in 350 msec 2023-07-24 20:10:46,450 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=17, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=Group_testCreateAndAssign execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-24 20:10:46,451 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCreateAndAssign","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690229446451"}]},"ts":"1690229446451"} 2023-07-24 20:10:46,453 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCreateAndAssign, state=ENABLED in hbase:meta 2023-07-24 20:10:46,456 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=17, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=Group_testCreateAndAssign execute state=CREATE_TABLE_POST_OPERATION 2023-07-24 20:10:46,459 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=17, state=SUCCESS; CreateTableProcedure table=Group_testCreateAndAssign in 969 msec 2023-07-24 20:10:46,608 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=17 2023-07-24 20:10:46,608 INFO [Listener at localhost/44145] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:Group_testCreateAndAssign, procId: 17 completed 2023-07-24 20:10:46,609 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:10:46,614 DEBUG [Listener at localhost/44145] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-24 20:10:46,616 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44130, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-24 20:10:46,618 DEBUG [Listener at localhost/44145] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-24 20:10:46,620 INFO [RS-EventLoopGroup-7-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51918, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-24 20:10:46,621 DEBUG [Listener at localhost/44145] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-24 20:10:46,623 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37866, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-24 20:10:46,624 DEBUG [Listener at localhost/44145] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-24 20:10:46,625 INFO [RS-EventLoopGroup-8-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60682, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-24 20:10:46,630 INFO [Listener at localhost/44145] client.HBaseAdmin$15(890): Started disable of Group_testCreateAndAssign 2023-07-24 20:10:46,634 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable Group_testCreateAndAssign 2023-07-24 20:10:46,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] procedure2.ProcedureExecutor(1029): Stored pid=20, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=Group_testCreateAndAssign 2023-07-24 20:10:46,646 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCreateAndAssign","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690229446646"}]},"ts":"1690229446646"} 2023-07-24 20:10:46,648 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=20 2023-07-24 20:10:46,648 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCreateAndAssign, state=DISABLING in hbase:meta 2023-07-24 20:10:46,650 INFO [PEWorker-5] procedure.DisableTableProcedure(293): Set Group_testCreateAndAssign to state=DISABLING 2023-07-24 20:10:46,651 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateAndAssign, region=cc526628eb03a103d976e6197331a7ce, UNASSIGN}] 2023-07-24 20:10:46,653 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=20, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateAndAssign, region=cc526628eb03a103d976e6197331a7ce, UNASSIGN 2023-07-24 20:10:46,655 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=cc526628eb03a103d976e6197331a7ce, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:46,655 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateAndAssign,,1690229445484.cc526628eb03a103d976e6197331a7ce.","families":{"info":[{"qualifier":"regioninfo","vlen":59,"tag":[],"timestamp":"1690229446655"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229446655"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229446655"}]},"ts":"1690229446655"} 2023-07-24 20:10:46,657 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure cc526628eb03a103d976e6197331a7ce, server=jenkins-hbase4.apache.org,45209,1690229445078}] 2023-07-24 20:10:46,749 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=20 2023-07-24 20:10:46,810 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close cc526628eb03a103d976e6197331a7ce 2023-07-24 20:10:46,811 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing cc526628eb03a103d976e6197331a7ce, disabling compactions & flushes 2023-07-24 20:10:46,812 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region Group_testCreateAndAssign,,1690229445484.cc526628eb03a103d976e6197331a7ce. 2023-07-24 20:10:46,812 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateAndAssign,,1690229445484.cc526628eb03a103d976e6197331a7ce. 2023-07-24 20:10:46,812 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on Group_testCreateAndAssign,,1690229445484.cc526628eb03a103d976e6197331a7ce. after waiting 0 ms 2023-07-24 20:10:46,812 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region Group_testCreateAndAssign,,1690229445484.cc526628eb03a103d976e6197331a7ce. 2023-07-24 20:10:46,817 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateAndAssign/cc526628eb03a103d976e6197331a7ce/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-24 20:10:46,818 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed Group_testCreateAndAssign,,1690229445484.cc526628eb03a103d976e6197331a7ce. 2023-07-24 20:10:46,818 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for cc526628eb03a103d976e6197331a7ce: 2023-07-24 20:10:46,820 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed cc526628eb03a103d976e6197331a7ce 2023-07-24 20:10:46,821 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=cc526628eb03a103d976e6197331a7ce, regionState=CLOSED 2023-07-24 20:10:46,821 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"Group_testCreateAndAssign,,1690229445484.cc526628eb03a103d976e6197331a7ce.","families":{"info":[{"qualifier":"regioninfo","vlen":59,"tag":[],"timestamp":"1690229446821"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229446821"}]},"ts":"1690229446821"} 2023-07-24 20:10:46,826 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-07-24 20:10:46,826 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure cc526628eb03a103d976e6197331a7ce, server=jenkins-hbase4.apache.org,45209,1690229445078 in 166 msec 2023-07-24 20:10:46,829 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-07-24 20:10:46,829 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateAndAssign, region=cc526628eb03a103d976e6197331a7ce, UNASSIGN in 175 msec 2023-07-24 20:10:46,830 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCreateAndAssign","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690229446830"}]},"ts":"1690229446830"} 2023-07-24 20:10:46,837 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCreateAndAssign, state=DISABLED in hbase:meta 2023-07-24 20:10:46,839 INFO [PEWorker-5] procedure.DisableTableProcedure(305): Set Group_testCreateAndAssign to state=DISABLED 2023-07-24 20:10:46,843 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=20, state=SUCCESS; DisableTableProcedure table=Group_testCreateAndAssign in 205 msec 2023-07-24 20:10:46,952 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=20 2023-07-24 20:10:46,952 INFO [Listener at localhost/44145] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:Group_testCreateAndAssign, procId: 20 completed 2023-07-24 20:10:46,959 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete Group_testCreateAndAssign 2023-07-24 20:10:46,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] procedure2.ProcedureExecutor(1029): Stored pid=23, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=Group_testCreateAndAssign 2023-07-24 20:10:46,972 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=23, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=Group_testCreateAndAssign 2023-07-24 20:10:46,973 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint(577): Removing deleted table 'Group_testCreateAndAssign' from rsgroup 'default' 2023-07-24 20:10:46,976 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=23, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=Group_testCreateAndAssign 2023-07-24 20:10:46,981 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:46,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:46,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-24 20:10:46,987 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateAndAssign/cc526628eb03a103d976e6197331a7ce 2023-07-24 20:10:46,990 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=23 2023-07-24 20:10:46,991 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateAndAssign/cc526628eb03a103d976e6197331a7ce/f, FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateAndAssign/cc526628eb03a103d976e6197331a7ce/recovered.edits] 2023-07-24 20:10:47,000 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateAndAssign/cc526628eb03a103d976e6197331a7ce/recovered.edits/4.seqid to hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/archive/data/default/Group_testCreateAndAssign/cc526628eb03a103d976e6197331a7ce/recovered.edits/4.seqid 2023-07-24 20:10:47,001 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateAndAssign/cc526628eb03a103d976e6197331a7ce 2023-07-24 20:10:47,001 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived Group_testCreateAndAssign regions 2023-07-24 20:10:47,007 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=23, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=Group_testCreateAndAssign 2023-07-24 20:10:47,028 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of Group_testCreateAndAssign from hbase:meta 2023-07-24 20:10:47,083 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'Group_testCreateAndAssign' descriptor. 2023-07-24 20:10:47,086 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=23, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=Group_testCreateAndAssign 2023-07-24 20:10:47,086 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'Group_testCreateAndAssign' from region states. 2023-07-24 20:10:47,087 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCreateAndAssign,,1690229445484.cc526628eb03a103d976e6197331a7ce.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690229447086"}]},"ts":"9223372036854775807"} 2023-07-24 20:10:47,089 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-07-24 20:10:47,089 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => cc526628eb03a103d976e6197331a7ce, NAME => 'Group_testCreateAndAssign,,1690229445484.cc526628eb03a103d976e6197331a7ce.', STARTKEY => '', ENDKEY => ''}] 2023-07-24 20:10:47,089 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'Group_testCreateAndAssign' as deleted. 2023-07-24 20:10:47,089 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCreateAndAssign","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1690229447089"}]},"ts":"9223372036854775807"} 2023-07-24 20:10:47,091 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=23 2023-07-24 20:10:47,092 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table Group_testCreateAndAssign state from META 2023-07-24 20:10:47,094 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=23, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=Group_testCreateAndAssign 2023-07-24 20:10:47,096 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, state=SUCCESS; DeleteTableProcedure table=Group_testCreateAndAssign in 133 msec 2023-07-24 20:10:47,292 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=23 2023-07-24 20:10:47,293 INFO [Listener at localhost/44145] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:Group_testCreateAndAssign, procId: 23 completed 2023-07-24 20:10:47,297 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:47,298 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:47,299 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-24 20:10:47,299 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-24 20:10:47,299 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-24 20:10:47,300 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-24 20:10:47,300 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-24 20:10:47,302 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-24 20:10:47,309 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:47,310 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-24 20:10:47,313 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-24 20:10:47,317 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase(152): Restoring servers: 0 2023-07-24 20:10:47,318 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-24 20:10:47,322 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:47,322 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:47,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-24 20:10:47,326 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-24 20:10:47,330 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:47,330 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:47,333 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:39247] to rsgroup master 2023-07-24 20:10:47,333 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-24 20:10:47,333 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.CallRunner(144): callId: 165 service: MasterService methodName: ExecMasterService size: 119 connection: 172.31.14.131:36610 deadline: 1690230647333, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. 2023-07-24 20:10:47,334 WARN [Listener at localhost/44145] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.afterMethod(TestRSGroupsBasics.java:82) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-24 20:10:47,337 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:10:47,338 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:47,338 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:47,339 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:34483, jenkins-hbase4.apache.org:34917, jenkins-hbase4.apache.org:39791, jenkins-hbase4.apache.org:45209], Tables:[hbase:meta, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-24 20:10:47,340 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-24 20:10:47,340 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-24 20:10:47,374 INFO [Listener at localhost/44145] hbase.ResourceChecker(175): after: rsgroup.TestRSGroupsBasics#testDefaultNamespaceCreateAndAssign Thread=493 (was 480) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-8 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-9 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2059508162_17 at /127.0.0.1:54072 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-1903532105-172.31.14.131-1690229433619:blk_1073741844_1020, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xb35bd70-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-1903532105-172.31.14.131-1690229433619:blk_1073741844_1020, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-8-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait0(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:182) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:302) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:366) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-7 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x229abe1d-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-1903532105-172.31.14.131-1690229433619:blk_1073741844_1020, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xb35bd70-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-6 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: AsyncFSWAL-0-hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4-prefix:jenkins-hbase4.apache.org,45209,1690229445078 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.4@localhost:37609 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2059508162_17 at /127.0.0.1:44918 [Receiving block BP-1903532105-172.31.14.131-1690229433619:blk_1073741844_1020] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2059508162_17 at /127.0.0.1:42952 [Receiving block BP-1903532105-172.31.14.131-1690229433619:blk_1073741844_1020] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2059508162_17 at /127.0.0.1:54084 [Receiving block BP-1903532105-172.31.14.131-1690229433619:blk_1073741844_1020] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=753 (was 723) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=394 (was 394), ProcessCount=176 (was 176), AvailableMemoryMB=5928 (was 5967) 2023-07-24 20:10:47,404 INFO [Listener at localhost/44145] hbase.ResourceChecker(147): before: rsgroup.TestRSGroupsBasics#testCreateMultiRegion Thread=493, OpenFileDescriptor=753, MaxFileDescriptor=60000, SystemLoadAverage=394, ProcessCount=176, AvailableMemoryMB=5926 2023-07-24 20:10:47,404 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase(132): testCreateMultiRegion 2023-07-24 20:10:47,416 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:47,418 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:47,433 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-24 20:10:47,433 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-24 20:10:47,433 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-24 20:10:47,435 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-24 20:10:47,435 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-24 20:10:47,436 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-24 20:10:47,441 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:47,441 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-24 20:10:47,443 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-24 20:10:47,448 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase(152): Restoring servers: 0 2023-07-24 20:10:47,450 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-24 20:10:47,453 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:47,454 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:47,456 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-24 20:10:47,468 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-24 20:10:47,476 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:47,477 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:47,481 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:39247] to rsgroup master 2023-07-24 20:10:47,481 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-24 20:10:47,481 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.CallRunner(144): callId: 193 service: MasterService methodName: ExecMasterService size: 119 connection: 172.31.14.131:36610 deadline: 1690230647480, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. 2023-07-24 20:10:47,482 WARN [Listener at localhost/44145] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.setUpBeforeMethod(TestRSGroupsBase.java:136) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.beforeMethod(TestRSGroupsBasics.java:77) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-24 20:10:47,484 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:10:47,486 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:47,486 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:47,487 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:34483, jenkins-hbase4.apache.org:34917, jenkins-hbase4.apache.org:39791, jenkins-hbase4.apache.org:45209], Tables:[hbase:meta, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-24 20:10:47,488 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-24 20:10:47,488 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-24 20:10:47,505 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'Group_testCreateMultiRegion', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-24 20:10:47,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=Group_testCreateMultiRegion 2023-07-24 20:10:47,511 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=24, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=Group_testCreateMultiRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-07-24 20:10:47,511 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "Group_testCreateMultiRegion" procId is: 24 2023-07-24 20:10:47,514 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=24 2023-07-24 20:10:47,516 DEBUG [PEWorker-4] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:47,517 DEBUG [PEWorker-4] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:47,517 DEBUG [PEWorker-4] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-24 20:10:47,521 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=24, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=Group_testCreateMultiRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-24 20:10:47,533 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/f55a12abc8e0bcbed2f6e2d3644e4a53 2023-07-24 20:10:47,533 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/7f557baf61410e30c82ed736835597ae 2023-07-24 20:10:47,533 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/486c039d584a40408ca57b59a9685da6 2023-07-24 20:10:47,533 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/2df6ec0be254267ea2d62569ac1c609d 2023-07-24 20:10:47,533 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/83406e51fcf8abcdba40d7c315de5777 2023-07-24 20:10:47,533 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/30fd729e25b9ba973f8749e1094e1710 2023-07-24 20:10:47,537 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/486c039d584a40408ca57b59a9685da6 empty. 2023-07-24 20:10:47,533 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/8a701529ae84652c9c029ed9bd063968 2023-07-24 20:10:47,533 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/98abe1672c7585d97343038205bd5450 2023-07-24 20:10:47,538 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/7f557baf61410e30c82ed736835597ae empty. 2023-07-24 20:10:47,536 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/f55a12abc8e0bcbed2f6e2d3644e4a53 empty. 2023-07-24 20:10:47,536 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/2df6ec0be254267ea2d62569ac1c609d empty. 2023-07-24 20:10:47,538 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/83406e51fcf8abcdba40d7c315de5777 empty. 2023-07-24 20:10:47,540 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/2df6ec0be254267ea2d62569ac1c609d 2023-07-24 20:10:47,540 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/437177da113d7ca44d1d74b0ae731139 2023-07-24 20:10:47,541 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/486c039d584a40408ca57b59a9685da6 2023-07-24 20:10:47,542 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/b3a80fc52bfa48de445ea8e679dbc14a 2023-07-24 20:10:47,542 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/98abe1672c7585d97343038205bd5450 empty. 2023-07-24 20:10:47,542 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/7f557baf61410e30c82ed736835597ae 2023-07-24 20:10:47,542 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/83406e51fcf8abcdba40d7c315de5777 2023-07-24 20:10:47,542 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/8a701529ae84652c9c029ed9bd063968 empty. 2023-07-24 20:10:47,543 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/30fd729e25b9ba973f8749e1094e1710 empty. 2023-07-24 20:10:47,543 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/437177da113d7ca44d1d74b0ae731139 empty. 2023-07-24 20:10:47,543 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/b3a80fc52bfa48de445ea8e679dbc14a empty. 2023-07-24 20:10:47,545 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/8a701529ae84652c9c029ed9bd063968 2023-07-24 20:10:47,545 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/30fd729e25b9ba973f8749e1094e1710 2023-07-24 20:10:47,546 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/437177da113d7ca44d1d74b0ae731139 2023-07-24 20:10:47,546 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/98abe1672c7585d97343038205bd5450 2023-07-24 20:10:47,546 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/f55a12abc8e0bcbed2f6e2d3644e4a53 2023-07-24 20:10:47,546 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/b3a80fc52bfa48de445ea8e679dbc14a 2023-07-24 20:10:47,546 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived Group_testCreateMultiRegion regions 2023-07-24 20:10:47,573 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/.tabledesc/.tableinfo.0000000001 2023-07-24 20:10:47,575 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(7675): creating {ENCODED => f55a12abc8e0bcbed2f6e2d3644e4a53, NAME => 'Group_testCreateMultiRegion,,1690229447504.f55a12abc8e0bcbed2f6e2d3644e4a53.', STARTKEY => '', ENDKEY => '\x00\x02\x04\x06\x08'}, tableDescriptor='Group_testCreateMultiRegion', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp 2023-07-24 20:10:47,576 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(7675): creating {ENCODED => 7f557baf61410e30c82ed736835597ae, NAME => 'Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1690229447504.7f557baf61410e30c82ed736835597ae.', STARTKEY => '\x00\x02\x04\x06\x08', ENDKEY => '\x00"$&('}, tableDescriptor='Group_testCreateMultiRegion', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp 2023-07-24 20:10:47,582 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(7675): creating {ENCODED => 486c039d584a40408ca57b59a9685da6, NAME => 'Group_testCreateMultiRegion,\x00"$&(,1690229447504.486c039d584a40408ca57b59a9685da6.', STARTKEY => '\x00"$&(', ENDKEY => '\x00BDFH'}, tableDescriptor='Group_testCreateMultiRegion', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp 2023-07-24 20:10:47,616 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=24 2023-07-24 20:10:47,645 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x00"$&(,1690229447504.486c039d584a40408ca57b59a9685da6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:47,646 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,,1690229447504.f55a12abc8e0bcbed2f6e2d3644e4a53.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:47,646 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1604): Closing 486c039d584a40408ca57b59a9685da6, disabling compactions & flushes 2023-07-24 20:10:47,646 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1604): Closing f55a12abc8e0bcbed2f6e2d3644e4a53, disabling compactions & flushes 2023-07-24 20:10:47,646 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,,1690229447504.f55a12abc8e0bcbed2f6e2d3644e4a53. 2023-07-24 20:10:47,646 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,,1690229447504.f55a12abc8e0bcbed2f6e2d3644e4a53. 2023-07-24 20:10:47,646 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,,1690229447504.f55a12abc8e0bcbed2f6e2d3644e4a53. after waiting 0 ms 2023-07-24 20:10:47,646 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,,1690229447504.f55a12abc8e0bcbed2f6e2d3644e4a53. 2023-07-24 20:10:47,646 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,,1690229447504.f55a12abc8e0bcbed2f6e2d3644e4a53. 2023-07-24 20:10:47,646 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x00"$&(,1690229447504.486c039d584a40408ca57b59a9685da6. 2023-07-24 20:10:47,647 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x00"$&(,1690229447504.486c039d584a40408ca57b59a9685da6. 2023-07-24 20:10:47,647 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x00"$&(,1690229447504.486c039d584a40408ca57b59a9685da6. after waiting 0 ms 2023-07-24 20:10:47,647 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x00"$&(,1690229447504.486c039d584a40408ca57b59a9685da6. 2023-07-24 20:10:47,647 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x00"$&(,1690229447504.486c039d584a40408ca57b59a9685da6. 2023-07-24 20:10:47,647 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1690229447504.7f557baf61410e30c82ed736835597ae.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:47,647 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1604): Closing 7f557baf61410e30c82ed736835597ae, disabling compactions & flushes 2023-07-24 20:10:47,647 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1690229447504.7f557baf61410e30c82ed736835597ae. 2023-07-24 20:10:47,647 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1690229447504.7f557baf61410e30c82ed736835597ae. 2023-07-24 20:10:47,647 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1690229447504.7f557baf61410e30c82ed736835597ae. after waiting 0 ms 2023-07-24 20:10:47,647 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1690229447504.7f557baf61410e30c82ed736835597ae. 2023-07-24 20:10:47,647 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1690229447504.7f557baf61410e30c82ed736835597ae. 2023-07-24 20:10:47,647 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1558): Region close journal for 7f557baf61410e30c82ed736835597ae: 2023-07-24 20:10:47,648 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(7675): creating {ENCODED => 83406e51fcf8abcdba40d7c315de5777, NAME => 'Group_testCreateMultiRegion,\x00BDFH,1690229447504.83406e51fcf8abcdba40d7c315de5777.', STARTKEY => '\x00BDFH', ENDKEY => '\x00bdfh'}, tableDescriptor='Group_testCreateMultiRegion', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp 2023-07-24 20:10:47,646 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1558): Region close journal for f55a12abc8e0bcbed2f6e2d3644e4a53: 2023-07-24 20:10:47,647 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1558): Region close journal for 486c039d584a40408ca57b59a9685da6: 2023-07-24 20:10:47,649 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(7675): creating {ENCODED => 98abe1672c7585d97343038205bd5450, NAME => 'Group_testCreateMultiRegion,\x00bdfh,1690229447504.98abe1672c7585d97343038205bd5450.', STARTKEY => '\x00bdfh', ENDKEY => '\x00\x82\x84\x86\x88'}, tableDescriptor='Group_testCreateMultiRegion', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp 2023-07-24 20:10:47,650 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(7675): creating {ENCODED => 8a701529ae84652c9c029ed9bd063968, NAME => 'Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1690229447504.8a701529ae84652c9c029ed9bd063968.', STARTKEY => '\x00\x82\x84\x86\x88', ENDKEY => '\x00\xA2\xA4\xA6\xA8'}, tableDescriptor='Group_testCreateMultiRegion', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp 2023-07-24 20:10:47,686 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x00BDFH,1690229447504.83406e51fcf8abcdba40d7c315de5777.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:47,686 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1604): Closing 83406e51fcf8abcdba40d7c315de5777, disabling compactions & flushes 2023-07-24 20:10:47,687 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x00BDFH,1690229447504.83406e51fcf8abcdba40d7c315de5777. 2023-07-24 20:10:47,687 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x00BDFH,1690229447504.83406e51fcf8abcdba40d7c315de5777. 2023-07-24 20:10:47,687 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x00BDFH,1690229447504.83406e51fcf8abcdba40d7c315de5777. after waiting 0 ms 2023-07-24 20:10:47,687 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x00BDFH,1690229447504.83406e51fcf8abcdba40d7c315de5777. 2023-07-24 20:10:47,687 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x00BDFH,1690229447504.83406e51fcf8abcdba40d7c315de5777. 2023-07-24 20:10:47,687 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1558): Region close journal for 83406e51fcf8abcdba40d7c315de5777: 2023-07-24 20:10:47,687 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(7675): creating {ENCODED => 2df6ec0be254267ea2d62569ac1c609d, NAME => 'Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1690229447504.2df6ec0be254267ea2d62569ac1c609d.', STARTKEY => '\x00\xA2\xA4\xA6\xA8', ENDKEY => '\x00\xC2\xC4\xC6\xC8'}, tableDescriptor='Group_testCreateMultiRegion', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp 2023-07-24 20:10:47,689 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1690229447504.8a701529ae84652c9c029ed9bd063968.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:47,690 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1604): Closing 8a701529ae84652c9c029ed9bd063968, disabling compactions & flushes 2023-07-24 20:10:47,690 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x00bdfh,1690229447504.98abe1672c7585d97343038205bd5450.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:47,691 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1690229447504.8a701529ae84652c9c029ed9bd063968. 2023-07-24 20:10:47,691 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1690229447504.8a701529ae84652c9c029ed9bd063968. 2023-07-24 20:10:47,692 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1690229447504.8a701529ae84652c9c029ed9bd063968. after waiting 0 ms 2023-07-24 20:10:47,692 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1690229447504.8a701529ae84652c9c029ed9bd063968. 2023-07-24 20:10:47,692 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1690229447504.8a701529ae84652c9c029ed9bd063968. 2023-07-24 20:10:47,692 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1558): Region close journal for 8a701529ae84652c9c029ed9bd063968: 2023-07-24 20:10:47,692 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(7675): creating {ENCODED => 30fd729e25b9ba973f8749e1094e1710, NAME => 'Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1690229447504.30fd729e25b9ba973f8749e1094e1710.', STARTKEY => '\x00\xC2\xC4\xC6\xC8', ENDKEY => '\x00\xE2\xE4\xE6\xE8'}, tableDescriptor='Group_testCreateMultiRegion', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp 2023-07-24 20:10:47,691 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1604): Closing 98abe1672c7585d97343038205bd5450, disabling compactions & flushes 2023-07-24 20:10:47,692 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x00bdfh,1690229447504.98abe1672c7585d97343038205bd5450. 2023-07-24 20:10:47,692 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x00bdfh,1690229447504.98abe1672c7585d97343038205bd5450. 2023-07-24 20:10:47,692 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x00bdfh,1690229447504.98abe1672c7585d97343038205bd5450. after waiting 0 ms 2023-07-24 20:10:47,693 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x00bdfh,1690229447504.98abe1672c7585d97343038205bd5450. 2023-07-24 20:10:47,693 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x00bdfh,1690229447504.98abe1672c7585d97343038205bd5450. 2023-07-24 20:10:47,693 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1558): Region close journal for 98abe1672c7585d97343038205bd5450: 2023-07-24 20:10:47,694 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(7675): creating {ENCODED => 437177da113d7ca44d1d74b0ae731139, NAME => 'Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1690229447504.437177da113d7ca44d1d74b0ae731139.', STARTKEY => '\x00\xE2\xE4\xE6\xE8', ENDKEY => '\x01\x03\x05\x07\x09'}, tableDescriptor='Group_testCreateMultiRegion', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp 2023-07-24 20:10:47,720 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1690229447504.2df6ec0be254267ea2d62569ac1c609d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:47,721 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1604): Closing 2df6ec0be254267ea2d62569ac1c609d, disabling compactions & flushes 2023-07-24 20:10:47,721 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1690229447504.2df6ec0be254267ea2d62569ac1c609d. 2023-07-24 20:10:47,721 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1690229447504.2df6ec0be254267ea2d62569ac1c609d. 2023-07-24 20:10:47,721 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1690229447504.2df6ec0be254267ea2d62569ac1c609d. after waiting 0 ms 2023-07-24 20:10:47,721 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1690229447504.2df6ec0be254267ea2d62569ac1c609d. 2023-07-24 20:10:47,721 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1690229447504.2df6ec0be254267ea2d62569ac1c609d. 2023-07-24 20:10:47,721 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1558): Region close journal for 2df6ec0be254267ea2d62569ac1c609d: 2023-07-24 20:10:47,722 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(7675): creating {ENCODED => b3a80fc52bfa48de445ea8e679dbc14a, NAME => 'Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1690229447504.b3a80fc52bfa48de445ea8e679dbc14a.', STARTKEY => '\x01\x03\x05\x07\x09', ENDKEY => ''}, tableDescriptor='Group_testCreateMultiRegion', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp 2023-07-24 20:10:47,725 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1690229447504.30fd729e25b9ba973f8749e1094e1710.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:47,726 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1604): Closing 30fd729e25b9ba973f8749e1094e1710, disabling compactions & flushes 2023-07-24 20:10:47,726 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1690229447504.30fd729e25b9ba973f8749e1094e1710. 2023-07-24 20:10:47,726 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1690229447504.30fd729e25b9ba973f8749e1094e1710. 2023-07-24 20:10:47,726 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1690229447504.30fd729e25b9ba973f8749e1094e1710. after waiting 0 ms 2023-07-24 20:10:47,726 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1690229447504.30fd729e25b9ba973f8749e1094e1710. 2023-07-24 20:10:47,726 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1690229447504.30fd729e25b9ba973f8749e1094e1710. 2023-07-24 20:10:47,726 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-2] regionserver.HRegion(1558): Region close journal for 30fd729e25b9ba973f8749e1094e1710: 2023-07-24 20:10:47,727 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1690229447504.437177da113d7ca44d1d74b0ae731139.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:47,728 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1604): Closing 437177da113d7ca44d1d74b0ae731139, disabling compactions & flushes 2023-07-24 20:10:47,728 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1690229447504.437177da113d7ca44d1d74b0ae731139. 2023-07-24 20:10:47,728 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1690229447504.437177da113d7ca44d1d74b0ae731139. 2023-07-24 20:10:47,728 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1690229447504.437177da113d7ca44d1d74b0ae731139. after waiting 0 ms 2023-07-24 20:10:47,728 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1690229447504.437177da113d7ca44d1d74b0ae731139. 2023-07-24 20:10:47,728 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1690229447504.437177da113d7ca44d1d74b0ae731139. 2023-07-24 20:10:47,728 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-0] regionserver.HRegion(1558): Region close journal for 437177da113d7ca44d1d74b0ae731139: 2023-07-24 20:10:47,759 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1690229447504.b3a80fc52bfa48de445ea8e679dbc14a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:47,759 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1604): Closing b3a80fc52bfa48de445ea8e679dbc14a, disabling compactions & flushes 2023-07-24 20:10:47,759 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1690229447504.b3a80fc52bfa48de445ea8e679dbc14a. 2023-07-24 20:10:47,759 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1690229447504.b3a80fc52bfa48de445ea8e679dbc14a. 2023-07-24 20:10:47,759 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1690229447504.b3a80fc52bfa48de445ea8e679dbc14a. after waiting 0 ms 2023-07-24 20:10:47,759 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1690229447504.b3a80fc52bfa48de445ea8e679dbc14a. 2023-07-24 20:10:47,759 INFO [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1690229447504.b3a80fc52bfa48de445ea8e679dbc14a. 2023-07-24 20:10:47,760 DEBUG [RegionOpenAndInit-Group_testCreateMultiRegion-pool-1] regionserver.HRegion(1558): Region close journal for b3a80fc52bfa48de445ea8e679dbc14a: 2023-07-24 20:10:47,763 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=24, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=Group_testCreateMultiRegion execute state=CREATE_TABLE_ADD_TO_META 2023-07-24 20:10:47,765 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x00\\x02\\x04\\x06\\x08,1690229447504.7f557baf61410e30c82ed736835597ae.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229447765"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229447765"}]},"ts":"1690229447765"} 2023-07-24 20:10:47,765 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,,1690229447504.f55a12abc8e0bcbed2f6e2d3644e4a53.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1690229447765"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229447765"}]},"ts":"1690229447765"} 2023-07-24 20:10:47,765 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x00\"$\u0026(,1690229447504.486c039d584a40408ca57b59a9685da6.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229447765"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229447765"}]},"ts":"1690229447765"} 2023-07-24 20:10:47,765 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x00BDFH,1690229447504.83406e51fcf8abcdba40d7c315de5777.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229447765"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229447765"}]},"ts":"1690229447765"} 2023-07-24 20:10:47,766 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x00\\x82\\x84\\x86\\x88,1690229447504.8a701529ae84652c9c029ed9bd063968.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229447765"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229447765"}]},"ts":"1690229447765"} 2023-07-24 20:10:47,766 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x00bdfh,1690229447504.98abe1672c7585d97343038205bd5450.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229447765"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229447765"}]},"ts":"1690229447765"} 2023-07-24 20:10:47,766 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x00\\xA2\\xA4\\xA6\\xA8,1690229447504.2df6ec0be254267ea2d62569ac1c609d.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229447765"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229447765"}]},"ts":"1690229447765"} 2023-07-24 20:10:47,766 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x00\\xC2\\xC4\\xC6\\xC8,1690229447504.30fd729e25b9ba973f8749e1094e1710.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229447765"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229447765"}]},"ts":"1690229447765"} 2023-07-24 20:10:47,766 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x00\\xE2\\xE4\\xE6\\xE8,1690229447504.437177da113d7ca44d1d74b0ae731139.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229447765"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229447765"}]},"ts":"1690229447765"} 2023-07-24 20:10:47,766 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x01\\x03\\x05\\x07\\x09,1690229447504.b3a80fc52bfa48de445ea8e679dbc14a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1690229447765"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229447765"}]},"ts":"1690229447765"} 2023-07-24 20:10:47,771 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 10 regions to meta. 2023-07-24 20:10:47,772 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=24, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=Group_testCreateMultiRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-24 20:10:47,772 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCreateMultiRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690229447772"}]},"ts":"1690229447772"} 2023-07-24 20:10:47,774 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCreateMultiRegion, state=ENABLING in hbase:meta 2023-07-24 20:10:47,777 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-24 20:10:47,777 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 20:10:47,777 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 20:10:47,777 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 20:10:47,777 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-24 20:10:47,777 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 20:10:47,778 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=24, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=f55a12abc8e0bcbed2f6e2d3644e4a53, ASSIGN}, {pid=26, ppid=24, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=7f557baf61410e30c82ed736835597ae, ASSIGN}, {pid=27, ppid=24, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=486c039d584a40408ca57b59a9685da6, ASSIGN}, {pid=28, ppid=24, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=83406e51fcf8abcdba40d7c315de5777, ASSIGN}, {pid=29, ppid=24, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=98abe1672c7585d97343038205bd5450, ASSIGN}, {pid=30, ppid=24, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=8a701529ae84652c9c029ed9bd063968, ASSIGN}, {pid=31, ppid=24, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=2df6ec0be254267ea2d62569ac1c609d, ASSIGN}, {pid=32, ppid=24, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=30fd729e25b9ba973f8749e1094e1710, ASSIGN}, {pid=33, ppid=24, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=437177da113d7ca44d1d74b0ae731139, ASSIGN}, {pid=34, ppid=24, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=b3a80fc52bfa48de445ea8e679dbc14a, ASSIGN}] 2023-07-24 20:10:47,781 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=27, ppid=24, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=486c039d584a40408ca57b59a9685da6, ASSIGN 2023-07-24 20:10:47,781 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=24, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=7f557baf61410e30c82ed736835597ae, ASSIGN 2023-07-24 20:10:47,782 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=24, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=83406e51fcf8abcdba40d7c315de5777, ASSIGN 2023-07-24 20:10:47,783 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=25, ppid=24, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=f55a12abc8e0bcbed2f6e2d3644e4a53, ASSIGN 2023-07-24 20:10:47,784 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=27, ppid=24, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=486c039d584a40408ca57b59a9685da6, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39791,1690229440084; forceNewPlan=false, retain=false 2023-07-24 20:10:47,785 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=24, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=7f557baf61410e30c82ed736835597ae, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34483,1690229439858; forceNewPlan=false, retain=false 2023-07-24 20:10:47,785 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=24, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=83406e51fcf8abcdba40d7c315de5777, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45209,1690229445078; forceNewPlan=false, retain=false 2023-07-24 20:10:47,787 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=34, ppid=24, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=b3a80fc52bfa48de445ea8e679dbc14a, ASSIGN 2023-07-24 20:10:47,787 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=33, ppid=24, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=437177da113d7ca44d1d74b0ae731139, ASSIGN 2023-07-24 20:10:47,787 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=25, ppid=24, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=f55a12abc8e0bcbed2f6e2d3644e4a53, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34917,1690229443609; forceNewPlan=false, retain=false 2023-07-24 20:10:47,788 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=24, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=30fd729e25b9ba973f8749e1094e1710, ASSIGN 2023-07-24 20:10:47,788 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=24, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=2df6ec0be254267ea2d62569ac1c609d, ASSIGN 2023-07-24 20:10:47,788 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=34, ppid=24, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=b3a80fc52bfa48de445ea8e679dbc14a, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34917,1690229443609; forceNewPlan=false, retain=false 2023-07-24 20:10:47,789 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=33, ppid=24, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=437177da113d7ca44d1d74b0ae731139, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34483,1690229439858; forceNewPlan=false, retain=false 2023-07-24 20:10:47,789 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=24, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=8a701529ae84652c9c029ed9bd063968, ASSIGN 2023-07-24 20:10:47,789 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=24, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=30fd729e25b9ba973f8749e1094e1710, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45209,1690229445078; forceNewPlan=false, retain=false 2023-07-24 20:10:47,790 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=24, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=98abe1672c7585d97343038205bd5450, ASSIGN 2023-07-24 20:10:47,790 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=24, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=2df6ec0be254267ea2d62569ac1c609d, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,39791,1690229440084; forceNewPlan=false, retain=false 2023-07-24 20:10:47,791 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=24, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=8a701529ae84652c9c029ed9bd063968, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34483,1690229439858; forceNewPlan=false, retain=false 2023-07-24 20:10:47,797 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=29, ppid=24, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=98abe1672c7585d97343038205bd5450, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34917,1690229443609; forceNewPlan=false, retain=false 2023-07-24 20:10:47,826 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=24 2023-07-24 20:10:47,934 INFO [jenkins-hbase4:39247] balancer.BaseLoadBalancer(1545): Reassigned 10 regions. 10 retained the pre-restart assignment. 2023-07-24 20:10:47,940 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=30fd729e25b9ba973f8749e1094e1710, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:47,940 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=83406e51fcf8abcdba40d7c315de5777, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:47,940 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=27 updating hbase:meta row=486c039d584a40408ca57b59a9685da6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:47,940 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=34 updating hbase:meta row=b3a80fc52bfa48de445ea8e679dbc14a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:47,940 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x00\\xC2\\xC4\\xC6\\xC8,1690229447504.30fd729e25b9ba973f8749e1094e1710.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229447940"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229447940"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229447940"}]},"ts":"1690229447940"} 2023-07-24 20:10:47,940 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x00\"$\u0026(,1690229447504.486c039d584a40408ca57b59a9685da6.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229447940"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229447940"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229447940"}]},"ts":"1690229447940"} 2023-07-24 20:10:47,940 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x01\\x03\\x05\\x07\\x09,1690229447504.b3a80fc52bfa48de445ea8e679dbc14a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1690229447940"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229447940"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229447940"}]},"ts":"1690229447940"} 2023-07-24 20:10:47,940 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=2df6ec0be254267ea2d62569ac1c609d, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:47,940 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x00BDFH,1690229447504.83406e51fcf8abcdba40d7c315de5777.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229447940"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229447940"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229447940"}]},"ts":"1690229447940"} 2023-07-24 20:10:47,940 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-24 20:10:47,940 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x00\\xA2\\xA4\\xA6\\xA8,1690229447504.2df6ec0be254267ea2d62569ac1c609d.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229447940"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229447940"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229447940"}]},"ts":"1690229447940"} 2023-07-24 20:10:47,943 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=35, ppid=32, state=RUNNABLE; OpenRegionProcedure 30fd729e25b9ba973f8749e1094e1710, server=jenkins-hbase4.apache.org,45209,1690229445078}] 2023-07-24 20:10:47,944 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=34, state=RUNNABLE; OpenRegionProcedure b3a80fc52bfa48de445ea8e679dbc14a, server=jenkins-hbase4.apache.org,34917,1690229443609}] 2023-07-24 20:10:47,946 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=37, ppid=27, state=RUNNABLE; OpenRegionProcedure 486c039d584a40408ca57b59a9685da6, server=jenkins-hbase4.apache.org,39791,1690229440084}] 2023-07-24 20:10:47,948 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=28, state=RUNNABLE; OpenRegionProcedure 83406e51fcf8abcdba40d7c315de5777, server=jenkins-hbase4.apache.org,45209,1690229445078}] 2023-07-24 20:10:47,949 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=31, state=RUNNABLE; OpenRegionProcedure 2df6ec0be254267ea2d62569ac1c609d, server=jenkins-hbase4.apache.org,39791,1690229440084}] 2023-07-24 20:10:47,951 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=98abe1672c7585d97343038205bd5450, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:47,951 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x00bdfh,1690229447504.98abe1672c7585d97343038205bd5450.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229447950"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229447950"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229447950"}]},"ts":"1690229447950"} 2023-07-24 20:10:47,951 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=25 updating hbase:meta row=f55a12abc8e0bcbed2f6e2d3644e4a53, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:47,951 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,,1690229447504.f55a12abc8e0bcbed2f6e2d3644e4a53.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1690229447951"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229447951"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229447951"}]},"ts":"1690229447951"} 2023-07-24 20:10:47,956 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=40, ppid=29, state=RUNNABLE; OpenRegionProcedure 98abe1672c7585d97343038205bd5450, server=jenkins-hbase4.apache.org,34917,1690229443609}] 2023-07-24 20:10:47,956 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=41, ppid=25, state=RUNNABLE; OpenRegionProcedure f55a12abc8e0bcbed2f6e2d3644e4a53, server=jenkins-hbase4.apache.org,34917,1690229443609}] 2023-07-24 20:10:47,957 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=33 updating hbase:meta row=437177da113d7ca44d1d74b0ae731139, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:47,957 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=8a701529ae84652c9c029ed9bd063968, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:47,957 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x00\\xE2\\xE4\\xE6\\xE8,1690229447504.437177da113d7ca44d1d74b0ae731139.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229447957"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229447957"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229447957"}]},"ts":"1690229447957"} 2023-07-24 20:10:47,958 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x00\\x82\\x84\\x86\\x88,1690229447504.8a701529ae84652c9c029ed9bd063968.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229447957"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229447957"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229447957"}]},"ts":"1690229447957"} 2023-07-24 20:10:47,959 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=7f557baf61410e30c82ed736835597ae, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:47,959 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x00\\x02\\x04\\x06\\x08,1690229447504.7f557baf61410e30c82ed736835597ae.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229447959"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229447959"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229447959"}]},"ts":"1690229447959"} 2023-07-24 20:10:47,960 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=33, state=RUNNABLE; OpenRegionProcedure 437177da113d7ca44d1d74b0ae731139, server=jenkins-hbase4.apache.org,34483,1690229439858}] 2023-07-24 20:10:47,961 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=30, state=RUNNABLE; OpenRegionProcedure 8a701529ae84652c9c029ed9bd063968, server=jenkins-hbase4.apache.org,34483,1690229439858}] 2023-07-24 20:10:47,962 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=44, ppid=26, state=RUNNABLE; OpenRegionProcedure 7f557baf61410e30c82ed736835597ae, server=jenkins-hbase4.apache.org,34483,1690229439858}] 2023-07-24 20:10:48,027 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: Master,sub=Coprocessor.Master.CP_org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase$CPMasterObserver 2023-07-24 20:10:48,027 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering Master,sub=Coprocessor.Master.CP_org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase$CPMasterObserver Metrics about HBase MasterObservers 2023-07-24 20:10:48,028 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-24 20:10:48,028 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-07-24 20:10:48,028 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: Master,sub=Coprocessor.Master.CP_org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint 2023-07-24 20:10:48,028 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering Master,sub=Coprocessor.Master.CP_org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint Metrics about HBase MasterObservers 2023-07-24 20:10:48,029 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-24 20:10:48,029 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:rsgroup' 2023-07-24 20:10:48,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open Group_testCreateMultiRegion,\x00BDFH,1690229447504.83406e51fcf8abcdba40d7c315de5777. 2023-07-24 20:10:48,101 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 83406e51fcf8abcdba40d7c315de5777, NAME => 'Group_testCreateMultiRegion,\x00BDFH,1690229447504.83406e51fcf8abcdba40d7c315de5777.', STARTKEY => '\x00BDFH', ENDKEY => '\x00bdfh'} 2023-07-24 20:10:48,101 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table Group_testCreateMultiRegion 83406e51fcf8abcdba40d7c315de5777 2023-07-24 20:10:48,102 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x00BDFH,1690229447504.83406e51fcf8abcdba40d7c315de5777.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:48,102 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 83406e51fcf8abcdba40d7c315de5777 2023-07-24 20:10:48,102 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 83406e51fcf8abcdba40d7c315de5777 2023-07-24 20:10:48,102 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1690229447504.b3a80fc52bfa48de445ea8e679dbc14a. 2023-07-24 20:10:48,102 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b3a80fc52bfa48de445ea8e679dbc14a, NAME => 'Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1690229447504.b3a80fc52bfa48de445ea8e679dbc14a.', STARTKEY => '\x01\x03\x05\x07\x09', ENDKEY => ''} 2023-07-24 20:10:48,103 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table Group_testCreateMultiRegion b3a80fc52bfa48de445ea8e679dbc14a 2023-07-24 20:10:48,103 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1690229447504.b3a80fc52bfa48de445ea8e679dbc14a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:48,103 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b3a80fc52bfa48de445ea8e679dbc14a 2023-07-24 20:10:48,103 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b3a80fc52bfa48de445ea8e679dbc14a 2023-07-24 20:10:48,104 INFO [StoreOpener-83406e51fcf8abcdba40d7c315de5777-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f of region 83406e51fcf8abcdba40d7c315de5777 2023-07-24 20:10:48,106 DEBUG [StoreOpener-83406e51fcf8abcdba40d7c315de5777-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/83406e51fcf8abcdba40d7c315de5777/f 2023-07-24 20:10:48,106 DEBUG [StoreOpener-83406e51fcf8abcdba40d7c315de5777-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/83406e51fcf8abcdba40d7c315de5777/f 2023-07-24 20:10:48,107 INFO [StoreOpener-b3a80fc52bfa48de445ea8e679dbc14a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f of region b3a80fc52bfa48de445ea8e679dbc14a 2023-07-24 20:10:48,107 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1690229447504.2df6ec0be254267ea2d62569ac1c609d. 2023-07-24 20:10:48,107 INFO [StoreOpener-83406e51fcf8abcdba40d7c315de5777-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 83406e51fcf8abcdba40d7c315de5777 columnFamilyName f 2023-07-24 20:10:48,107 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2df6ec0be254267ea2d62569ac1c609d, NAME => 'Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1690229447504.2df6ec0be254267ea2d62569ac1c609d.', STARTKEY => '\x00\xA2\xA4\xA6\xA8', ENDKEY => '\x00\xC2\xC4\xC6\xC8'} 2023-07-24 20:10:48,107 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table Group_testCreateMultiRegion 2df6ec0be254267ea2d62569ac1c609d 2023-07-24 20:10:48,107 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1690229447504.2df6ec0be254267ea2d62569ac1c609d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:48,107 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2df6ec0be254267ea2d62569ac1c609d 2023-07-24 20:10:48,107 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2df6ec0be254267ea2d62569ac1c609d 2023-07-24 20:10:48,107 INFO [StoreOpener-83406e51fcf8abcdba40d7c315de5777-1] regionserver.HStore(310): Store=83406e51fcf8abcdba40d7c315de5777/f, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:10:48,109 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/83406e51fcf8abcdba40d7c315de5777 2023-07-24 20:10:48,109 DEBUG [StoreOpener-b3a80fc52bfa48de445ea8e679dbc14a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/b3a80fc52bfa48de445ea8e679dbc14a/f 2023-07-24 20:10:48,110 DEBUG [StoreOpener-b3a80fc52bfa48de445ea8e679dbc14a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/b3a80fc52bfa48de445ea8e679dbc14a/f 2023-07-24 20:10:48,110 INFO [StoreOpener-2df6ec0be254267ea2d62569ac1c609d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f of region 2df6ec0be254267ea2d62569ac1c609d 2023-07-24 20:10:48,110 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/83406e51fcf8abcdba40d7c315de5777 2023-07-24 20:10:48,110 INFO [StoreOpener-b3a80fc52bfa48de445ea8e679dbc14a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b3a80fc52bfa48de445ea8e679dbc14a columnFamilyName f 2023-07-24 20:10:48,111 INFO [StoreOpener-b3a80fc52bfa48de445ea8e679dbc14a-1] regionserver.HStore(310): Store=b3a80fc52bfa48de445ea8e679dbc14a/f, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:10:48,112 DEBUG [StoreOpener-2df6ec0be254267ea2d62569ac1c609d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/2df6ec0be254267ea2d62569ac1c609d/f 2023-07-24 20:10:48,113 DEBUG [StoreOpener-2df6ec0be254267ea2d62569ac1c609d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/2df6ec0be254267ea2d62569ac1c609d/f 2023-07-24 20:10:48,113 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/b3a80fc52bfa48de445ea8e679dbc14a 2023-07-24 20:10:48,113 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:48,113 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-24 20:10:48,114 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/b3a80fc52bfa48de445ea8e679dbc14a 2023-07-24 20:10:48,114 INFO [StoreOpener-2df6ec0be254267ea2d62569ac1c609d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2df6ec0be254267ea2d62569ac1c609d columnFamilyName f 2023-07-24 20:10:48,115 INFO [StoreOpener-2df6ec0be254267ea2d62569ac1c609d-1] regionserver.HStore(310): Store=2df6ec0be254267ea2d62569ac1c609d/f, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:10:48,115 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 83406e51fcf8abcdba40d7c315de5777 2023-07-24 20:10:48,116 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:44136, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-24 20:10:48,117 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/2df6ec0be254267ea2d62569ac1c609d 2023-07-24 20:10:48,117 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/2df6ec0be254267ea2d62569ac1c609d 2023-07-24 20:10:48,119 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/83406e51fcf8abcdba40d7c315de5777/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 20:10:48,119 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b3a80fc52bfa48de445ea8e679dbc14a 2023-07-24 20:10:48,120 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 83406e51fcf8abcdba40d7c315de5777; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=10858788160, jitterRate=0.011303454637527466}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 20:10:48,120 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 83406e51fcf8abcdba40d7c315de5777: 2023-07-24 20:10:48,123 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2df6ec0be254267ea2d62569ac1c609d 2023-07-24 20:10:48,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1690229447504.8a701529ae84652c9c029ed9bd063968. 2023-07-24 20:10:48,125 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8a701529ae84652c9c029ed9bd063968, NAME => 'Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1690229447504.8a701529ae84652c9c029ed9bd063968.', STARTKEY => '\x00\x82\x84\x86\x88', ENDKEY => '\x00\xA2\xA4\xA6\xA8'} 2023-07-24 20:10:48,125 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table Group_testCreateMultiRegion 8a701529ae84652c9c029ed9bd063968 2023-07-24 20:10:48,126 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1690229447504.8a701529ae84652c9c029ed9bd063968.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:48,126 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 8a701529ae84652c9c029ed9bd063968 2023-07-24 20:10:48,126 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 8a701529ae84652c9c029ed9bd063968 2023-07-24 20:10:48,126 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/b3a80fc52bfa48de445ea8e679dbc14a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 20:10:48,126 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/2df6ec0be254267ea2d62569ac1c609d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 20:10:48,127 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b3a80fc52bfa48de445ea8e679dbc14a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=10016468480, jitterRate=-0.06714367866516113}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 20:10:48,127 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b3a80fc52bfa48de445ea8e679dbc14a: 2023-07-24 20:10:48,127 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2df6ec0be254267ea2d62569ac1c609d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=9685442400, jitterRate=-0.09797288477420807}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 20:10:48,127 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2df6ec0be254267ea2d62569ac1c609d: 2023-07-24 20:10:48,128 INFO [StoreOpener-8a701529ae84652c9c029ed9bd063968-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f of region 8a701529ae84652c9c029ed9bd063968 2023-07-24 20:10:48,128 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=24 2023-07-24 20:10:48,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1690229447504.b3a80fc52bfa48de445ea8e679dbc14a., pid=36, masterSystemTime=1690229448098 2023-07-24 20:10:48,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1690229447504.2df6ec0be254267ea2d62569ac1c609d., pid=39, masterSystemTime=1690229448100 2023-07-24 20:10:48,130 DEBUG [StoreOpener-8a701529ae84652c9c029ed9bd063968-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/8a701529ae84652c9c029ed9bd063968/f 2023-07-24 20:10:48,130 DEBUG [StoreOpener-8a701529ae84652c9c029ed9bd063968-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/8a701529ae84652c9c029ed9bd063968/f 2023-07-24 20:10:48,130 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1690229447504.b3a80fc52bfa48de445ea8e679dbc14a. 2023-07-24 20:10:48,131 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1690229447504.b3a80fc52bfa48de445ea8e679dbc14a. 2023-07-24 20:10:48,131 INFO [StoreOpener-8a701529ae84652c9c029ed9bd063968-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8a701529ae84652c9c029ed9bd063968 columnFamilyName f 2023-07-24 20:10:48,131 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open Group_testCreateMultiRegion,,1690229447504.f55a12abc8e0bcbed2f6e2d3644e4a53. 2023-07-24 20:10:48,131 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f55a12abc8e0bcbed2f6e2d3644e4a53, NAME => 'Group_testCreateMultiRegion,,1690229447504.f55a12abc8e0bcbed2f6e2d3644e4a53.', STARTKEY => '', ENDKEY => '\x00\x02\x04\x06\x08'} 2023-07-24 20:10:48,131 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=34 updating hbase:meta row=b3a80fc52bfa48de445ea8e679dbc14a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:48,133 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1690229447504.2df6ec0be254267ea2d62569ac1c609d. 2023-07-24 20:10:48,133 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"Group_testCreateMultiRegion,\\x01\\x03\\x05\\x07\\x09,1690229447504.b3a80fc52bfa48de445ea8e679dbc14a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1690229448131"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690229448131"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690229448131"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690229448131"}]},"ts":"1690229448131"} 2023-07-24 20:10:48,131 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for Group_testCreateMultiRegion,\x00BDFH,1690229447504.83406e51fcf8abcdba40d7c315de5777., pid=38, masterSystemTime=1690229448096 2023-07-24 20:10:48,133 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1690229447504.2df6ec0be254267ea2d62569ac1c609d. 2023-07-24 20:10:48,133 INFO [StoreOpener-8a701529ae84652c9c029ed9bd063968-1] regionserver.HStore(310): Store=8a701529ae84652c9c029ed9bd063968/f, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:10:48,134 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open Group_testCreateMultiRegion,\x00"$&(,1690229447504.486c039d584a40408ca57b59a9685da6. 2023-07-24 20:10:48,131 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table Group_testCreateMultiRegion f55a12abc8e0bcbed2f6e2d3644e4a53 2023-07-24 20:10:48,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,,1690229447504.f55a12abc8e0bcbed2f6e2d3644e4a53.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:48,134 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for f55a12abc8e0bcbed2f6e2d3644e4a53 2023-07-24 20:10:48,135 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for f55a12abc8e0bcbed2f6e2d3644e4a53 2023-07-24 20:10:48,135 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=2df6ec0be254267ea2d62569ac1c609d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:48,135 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"Group_testCreateMultiRegion,\\x00\\xA2\\xA4\\xA6\\xA8,1690229447504.2df6ec0be254267ea2d62569ac1c609d.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229448135"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690229448135"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690229448135"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690229448135"}]},"ts":"1690229448135"} 2023-07-24 20:10:48,135 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 486c039d584a40408ca57b59a9685da6, NAME => 'Group_testCreateMultiRegion,\x00"$&(,1690229447504.486c039d584a40408ca57b59a9685da6.', STARTKEY => '\x00"$&(', ENDKEY => '\x00BDFH'} 2023-07-24 20:10:48,135 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/8a701529ae84652c9c029ed9bd063968 2023-07-24 20:10:48,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table Group_testCreateMultiRegion 486c039d584a40408ca57b59a9685da6 2023-07-24 20:10:48,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x00"$&(,1690229447504.486c039d584a40408ca57b59a9685da6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:48,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 486c039d584a40408ca57b59a9685da6 2023-07-24 20:10:48,137 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 486c039d584a40408ca57b59a9685da6 2023-07-24 20:10:48,137 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/8a701529ae84652c9c029ed9bd063968 2023-07-24 20:10:48,137 INFO [StoreOpener-f55a12abc8e0bcbed2f6e2d3644e4a53-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f of region f55a12abc8e0bcbed2f6e2d3644e4a53 2023-07-24 20:10:48,138 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for Group_testCreateMultiRegion,\x00BDFH,1690229447504.83406e51fcf8abcdba40d7c315de5777. 2023-07-24 20:10:48,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened Group_testCreateMultiRegion,\x00BDFH,1690229447504.83406e51fcf8abcdba40d7c315de5777. 2023-07-24 20:10:48,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1690229447504.30fd729e25b9ba973f8749e1094e1710. 2023-07-24 20:10:48,139 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=83406e51fcf8abcdba40d7c315de5777, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:48,139 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"Group_testCreateMultiRegion,\\x00BDFH,1690229447504.83406e51fcf8abcdba40d7c315de5777.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229448139"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690229448139"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690229448139"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690229448139"}]},"ts":"1690229448139"} 2023-07-24 20:10:48,140 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 30fd729e25b9ba973f8749e1094e1710, NAME => 'Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1690229447504.30fd729e25b9ba973f8749e1094e1710.', STARTKEY => '\x00\xC2\xC4\xC6\xC8', ENDKEY => '\x00\xE2\xE4\xE6\xE8'} 2023-07-24 20:10:48,141 DEBUG [StoreOpener-f55a12abc8e0bcbed2f6e2d3644e4a53-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/f55a12abc8e0bcbed2f6e2d3644e4a53/f 2023-07-24 20:10:48,150 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=24, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=b3a80fc52bfa48de445ea8e679dbc14a, ASSIGN in 366 msec 2023-07-24 20:10:48,144 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=34 2023-07-24 20:10:48,143 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 8a701529ae84652c9c029ed9bd063968 2023-07-24 20:10:48,153 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=34, state=SUCCESS; OpenRegionProcedure b3a80fc52bfa48de445ea8e679dbc14a, server=jenkins-hbase4.apache.org,34917,1690229443609 in 195 msec 2023-07-24 20:10:48,153 INFO [StoreOpener-486c039d584a40408ca57b59a9685da6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f of region 486c039d584a40408ca57b59a9685da6 2023-07-24 20:10:48,141 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table Group_testCreateMultiRegion 30fd729e25b9ba973f8749e1094e1710 2023-07-24 20:10:48,153 DEBUG [StoreOpener-f55a12abc8e0bcbed2f6e2d3644e4a53-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/f55a12abc8e0bcbed2f6e2d3644e4a53/f 2023-07-24 20:10:48,154 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1690229447504.30fd729e25b9ba973f8749e1094e1710.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:48,154 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 30fd729e25b9ba973f8749e1094e1710 2023-07-24 20:10:48,154 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 30fd729e25b9ba973f8749e1094e1710 2023-07-24 20:10:48,154 INFO [StoreOpener-f55a12abc8e0bcbed2f6e2d3644e4a53-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f55a12abc8e0bcbed2f6e2d3644e4a53 columnFamilyName f 2023-07-24 20:10:48,154 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=31 2023-07-24 20:10:48,155 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=31, state=SUCCESS; OpenRegionProcedure 2df6ec0be254267ea2d62569ac1c609d, server=jenkins-hbase4.apache.org,39791,1690229440084 in 194 msec 2023-07-24 20:10:48,155 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=28 2023-07-24 20:10:48,155 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=28, state=SUCCESS; OpenRegionProcedure 83406e51fcf8abcdba40d7c315de5777, server=jenkins-hbase4.apache.org,45209,1690229445078 in 195 msec 2023-07-24 20:10:48,155 INFO [StoreOpener-f55a12abc8e0bcbed2f6e2d3644e4a53-1] regionserver.HStore(310): Store=f55a12abc8e0bcbed2f6e2d3644e4a53/f, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:10:48,158 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=24, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=2df6ec0be254267ea2d62569ac1c609d, ASSIGN in 376 msec 2023-07-24 20:10:48,158 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/f55a12abc8e0bcbed2f6e2d3644e4a53 2023-07-24 20:10:48,158 DEBUG [StoreOpener-486c039d584a40408ca57b59a9685da6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/486c039d584a40408ca57b59a9685da6/f 2023-07-24 20:10:48,158 DEBUG [StoreOpener-486c039d584a40408ca57b59a9685da6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/486c039d584a40408ca57b59a9685da6/f 2023-07-24 20:10:48,159 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/f55a12abc8e0bcbed2f6e2d3644e4a53 2023-07-24 20:10:48,159 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=24, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=83406e51fcf8abcdba40d7c315de5777, ASSIGN in 377 msec 2023-07-24 20:10:48,159 INFO [StoreOpener-486c039d584a40408ca57b59a9685da6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 486c039d584a40408ca57b59a9685da6 columnFamilyName f 2023-07-24 20:10:48,160 INFO [StoreOpener-486c039d584a40408ca57b59a9685da6-1] regionserver.HStore(310): Store=486c039d584a40408ca57b59a9685da6/f, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:10:48,163 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for f55a12abc8e0bcbed2f6e2d3644e4a53 2023-07-24 20:10:48,176 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/8a701529ae84652c9c029ed9bd063968/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 20:10:48,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 8a701529ae84652c9c029ed9bd063968; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=10196388160, jitterRate=-0.05038735270500183}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 20:10:48,177 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 8a701529ae84652c9c029ed9bd063968: 2023-07-24 20:10:48,177 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/486c039d584a40408ca57b59a9685da6 2023-07-24 20:10:48,178 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/486c039d584a40408ca57b59a9685da6 2023-07-24 20:10:48,178 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/f55a12abc8e0bcbed2f6e2d3644e4a53/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 20:10:48,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened f55a12abc8e0bcbed2f6e2d3644e4a53; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=11079476000, jitterRate=0.031856611371040344}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 20:10:48,179 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for f55a12abc8e0bcbed2f6e2d3644e4a53: 2023-07-24 20:10:48,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for Group_testCreateMultiRegion,,1690229447504.f55a12abc8e0bcbed2f6e2d3644e4a53., pid=41, masterSystemTime=1690229448098 2023-07-24 20:10:48,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1690229447504.8a701529ae84652c9c029ed9bd063968., pid=43, masterSystemTime=1690229448113 2023-07-24 20:10:48,186 INFO [StoreOpener-30fd729e25b9ba973f8749e1094e1710-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f of region 30fd729e25b9ba973f8749e1094e1710 2023-07-24 20:10:48,189 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 486c039d584a40408ca57b59a9685da6 2023-07-24 20:10:48,189 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for Group_testCreateMultiRegion,,1690229447504.f55a12abc8e0bcbed2f6e2d3644e4a53. 2023-07-24 20:10:48,190 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened Group_testCreateMultiRegion,,1690229447504.f55a12abc8e0bcbed2f6e2d3644e4a53. 2023-07-24 20:10:48,190 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open Group_testCreateMultiRegion,\x00bdfh,1690229447504.98abe1672c7585d97343038205bd5450. 2023-07-24 20:10:48,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 98abe1672c7585d97343038205bd5450, NAME => 'Group_testCreateMultiRegion,\x00bdfh,1690229447504.98abe1672c7585d97343038205bd5450.', STARTKEY => '\x00bdfh', ENDKEY => '\x00\x82\x84\x86\x88'} 2023-07-24 20:10:48,190 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=25 updating hbase:meta row=f55a12abc8e0bcbed2f6e2d3644e4a53, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:48,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table Group_testCreateMultiRegion 98abe1672c7585d97343038205bd5450 2023-07-24 20:10:48,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x00bdfh,1690229447504.98abe1672c7585d97343038205bd5450.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:48,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 98abe1672c7585d97343038205bd5450 2023-07-24 20:10:48,191 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 98abe1672c7585d97343038205bd5450 2023-07-24 20:10:48,190 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"Group_testCreateMultiRegion,,1690229447504.f55a12abc8e0bcbed2f6e2d3644e4a53.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1690229448190"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690229448190"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690229448190"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690229448190"}]},"ts":"1690229448190"} 2023-07-24 20:10:48,191 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1690229447504.8a701529ae84652c9c029ed9bd063968. 2023-07-24 20:10:48,193 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1690229447504.8a701529ae84652c9c029ed9bd063968. 2023-07-24 20:10:48,193 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1690229447504.437177da113d7ca44d1d74b0ae731139. 2023-07-24 20:10:48,193 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 437177da113d7ca44d1d74b0ae731139, NAME => 'Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1690229447504.437177da113d7ca44d1d74b0ae731139.', STARTKEY => '\x00\xE2\xE4\xE6\xE8', ENDKEY => '\x01\x03\x05\x07\x09'} 2023-07-24 20:10:48,193 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table Group_testCreateMultiRegion 437177da113d7ca44d1d74b0ae731139 2023-07-24 20:10:48,193 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1690229447504.437177da113d7ca44d1d74b0ae731139.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:48,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 437177da113d7ca44d1d74b0ae731139 2023-07-24 20:10:48,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 437177da113d7ca44d1d74b0ae731139 2023-07-24 20:10:48,194 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=8a701529ae84652c9c029ed9bd063968, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:48,194 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"Group_testCreateMultiRegion,\\x00\\x82\\x84\\x86\\x88,1690229447504.8a701529ae84652c9c029ed9bd063968.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229448194"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690229448194"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690229448194"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690229448194"}]},"ts":"1690229448194"} 2023-07-24 20:10:48,196 DEBUG [StoreOpener-30fd729e25b9ba973f8749e1094e1710-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/30fd729e25b9ba973f8749e1094e1710/f 2023-07-24 20:10:48,196 DEBUG [StoreOpener-30fd729e25b9ba973f8749e1094e1710-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/30fd729e25b9ba973f8749e1094e1710/f 2023-07-24 20:10:48,196 INFO [StoreOpener-30fd729e25b9ba973f8749e1094e1710-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 30fd729e25b9ba973f8749e1094e1710 columnFamilyName f 2023-07-24 20:10:48,196 INFO [StoreOpener-98abe1672c7585d97343038205bd5450-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f of region 98abe1672c7585d97343038205bd5450 2023-07-24 20:10:48,198 INFO [StoreOpener-30fd729e25b9ba973f8749e1094e1710-1] regionserver.HStore(310): Store=30fd729e25b9ba973f8749e1094e1710/f, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:10:48,200 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/486c039d584a40408ca57b59a9685da6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 20:10:48,200 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=41, resume processing ppid=25 2023-07-24 20:10:48,201 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=41, ppid=25, state=SUCCESS; OpenRegionProcedure f55a12abc8e0bcbed2f6e2d3644e4a53, server=jenkins-hbase4.apache.org,34917,1690229443609 in 239 msec 2023-07-24 20:10:48,201 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/30fd729e25b9ba973f8749e1094e1710 2023-07-24 20:10:48,202 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/30fd729e25b9ba973f8749e1094e1710 2023-07-24 20:10:48,202 INFO [StoreOpener-437177da113d7ca44d1d74b0ae731139-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f of region 437177da113d7ca44d1d74b0ae731139 2023-07-24 20:10:48,203 DEBUG [StoreOpener-98abe1672c7585d97343038205bd5450-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/98abe1672c7585d97343038205bd5450/f 2023-07-24 20:10:48,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 486c039d584a40408ca57b59a9685da6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=11176711520, jitterRate=0.04091237485408783}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 20:10:48,204 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 486c039d584a40408ca57b59a9685da6: 2023-07-24 20:10:48,203 DEBUG [StoreOpener-98abe1672c7585d97343038205bd5450-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/98abe1672c7585d97343038205bd5450/f 2023-07-24 20:10:48,205 INFO [StoreOpener-98abe1672c7585d97343038205bd5450-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 98abe1672c7585d97343038205bd5450 columnFamilyName f 2023-07-24 20:10:48,205 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=24, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=f55a12abc8e0bcbed2f6e2d3644e4a53, ASSIGN in 422 msec 2023-07-24 20:10:48,206 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for Group_testCreateMultiRegion,\x00"$&(,1690229447504.486c039d584a40408ca57b59a9685da6., pid=37, masterSystemTime=1690229448100 2023-07-24 20:10:48,206 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=30 2023-07-24 20:10:48,206 DEBUG [StoreOpener-437177da113d7ca44d1d74b0ae731139-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/437177da113d7ca44d1d74b0ae731139/f 2023-07-24 20:10:48,206 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=30, state=SUCCESS; OpenRegionProcedure 8a701529ae84652c9c029ed9bd063968, server=jenkins-hbase4.apache.org,34483,1690229439858 in 237 msec 2023-07-24 20:10:48,206 INFO [StoreOpener-98abe1672c7585d97343038205bd5450-1] regionserver.HStore(310): Store=98abe1672c7585d97343038205bd5450/f, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:10:48,206 DEBUG [StoreOpener-437177da113d7ca44d1d74b0ae731139-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/437177da113d7ca44d1d74b0ae731139/f 2023-07-24 20:10:48,207 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/98abe1672c7585d97343038205bd5450 2023-07-24 20:10:48,208 INFO [StoreOpener-437177da113d7ca44d1d74b0ae731139-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 437177da113d7ca44d1d74b0ae731139 columnFamilyName f 2023-07-24 20:10:48,208 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/98abe1672c7585d97343038205bd5450 2023-07-24 20:10:48,209 INFO [StoreOpener-437177da113d7ca44d1d74b0ae731139-1] regionserver.HStore(310): Store=437177da113d7ca44d1d74b0ae731139/f, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:10:48,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 30fd729e25b9ba973f8749e1094e1710 2023-07-24 20:10:48,209 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=24, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=8a701529ae84652c9c029ed9bd063968, ASSIGN in 428 msec 2023-07-24 20:10:48,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for Group_testCreateMultiRegion,\x00"$&(,1690229447504.486c039d584a40408ca57b59a9685da6. 2023-07-24 20:10:48,210 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened Group_testCreateMultiRegion,\x00"$&(,1690229447504.486c039d584a40408ca57b59a9685da6. 2023-07-24 20:10:48,210 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=27 updating hbase:meta row=486c039d584a40408ca57b59a9685da6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:48,210 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"Group_testCreateMultiRegion,\\x00\"$\u0026(,1690229447504.486c039d584a40408ca57b59a9685da6.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229448210"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690229448210"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690229448210"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690229448210"}]},"ts":"1690229448210"} 2023-07-24 20:10:48,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/437177da113d7ca44d1d74b0ae731139 2023-07-24 20:10:48,212 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/437177da113d7ca44d1d74b0ae731139 2023-07-24 20:10:48,214 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/30fd729e25b9ba973f8749e1094e1710/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 20:10:48,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 30fd729e25b9ba973f8749e1094e1710; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=10393091520, jitterRate=-0.0320679247379303}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 20:10:48,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 30fd729e25b9ba973f8749e1094e1710: 2023-07-24 20:10:48,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1690229447504.30fd729e25b9ba973f8749e1094e1710., pid=35, masterSystemTime=1690229448096 2023-07-24 20:10:48,217 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=27 2023-07-24 20:10:48,217 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 98abe1672c7585d97343038205bd5450 2023-07-24 20:10:48,217 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=27, state=SUCCESS; OpenRegionProcedure 486c039d584a40408ca57b59a9685da6, server=jenkins-hbase4.apache.org,39791,1690229440084 in 266 msec 2023-07-24 20:10:48,217 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 437177da113d7ca44d1d74b0ae731139 2023-07-24 20:10:48,219 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=24, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=486c039d584a40408ca57b59a9685da6, ASSIGN in 439 msec 2023-07-24 20:10:48,220 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1690229447504.30fd729e25b9ba973f8749e1094e1710. 2023-07-24 20:10:48,220 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1690229447504.30fd729e25b9ba973f8749e1094e1710. 2023-07-24 20:10:48,220 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=30fd729e25b9ba973f8749e1094e1710, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:48,220 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"Group_testCreateMultiRegion,\\x00\\xC2\\xC4\\xC6\\xC8,1690229447504.30fd729e25b9ba973f8749e1094e1710.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229448220"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690229448220"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690229448220"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690229448220"}]},"ts":"1690229448220"} 2023-07-24 20:10:48,230 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/437177da113d7ca44d1d74b0ae731139/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 20:10:48,231 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/98abe1672c7585d97343038205bd5450/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 20:10:48,231 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 437177da113d7ca44d1d74b0ae731139; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=11374643680, jitterRate=0.05934624373912811}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 20:10:48,231 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 437177da113d7ca44d1d74b0ae731139: 2023-07-24 20:10:48,238 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1690229447504.437177da113d7ca44d1d74b0ae731139., pid=42, masterSystemTime=1690229448113 2023-07-24 20:10:48,238 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 98abe1672c7585d97343038205bd5450; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=11668363520, jitterRate=0.08670103549957275}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 20:10:48,239 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 98abe1672c7585d97343038205bd5450: 2023-07-24 20:10:48,239 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=35, resume processing ppid=32 2023-07-24 20:10:48,239 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, ppid=32, state=SUCCESS; OpenRegionProcedure 30fd729e25b9ba973f8749e1094e1710, server=jenkins-hbase4.apache.org,45209,1690229445078 in 281 msec 2023-07-24 20:10:48,240 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for Group_testCreateMultiRegion,\x00bdfh,1690229447504.98abe1672c7585d97343038205bd5450., pid=40, masterSystemTime=1690229448098 2023-07-24 20:10:48,241 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1690229447504.437177da113d7ca44d1d74b0ae731139. 2023-07-24 20:10:48,241 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1690229447504.437177da113d7ca44d1d74b0ae731139. 2023-07-24 20:10:48,241 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1690229447504.7f557baf61410e30c82ed736835597ae. 2023-07-24 20:10:48,241 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 7f557baf61410e30c82ed736835597ae, NAME => 'Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1690229447504.7f557baf61410e30c82ed736835597ae.', STARTKEY => '\x00\x02\x04\x06\x08', ENDKEY => '\x00"$&('} 2023-07-24 20:10:48,241 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table Group_testCreateMultiRegion 7f557baf61410e30c82ed736835597ae 2023-07-24 20:10:48,242 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1690229447504.7f557baf61410e30c82ed736835597ae.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:48,242 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 7f557baf61410e30c82ed736835597ae 2023-07-24 20:10:48,242 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 7f557baf61410e30c82ed736835597ae 2023-07-24 20:10:48,243 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=24, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=30fd729e25b9ba973f8749e1094e1710, ASSIGN in 461 msec 2023-07-24 20:10:48,243 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=33 updating hbase:meta row=437177da113d7ca44d1d74b0ae731139, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:48,243 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"Group_testCreateMultiRegion,\\x00\\xE2\\xE4\\xE6\\xE8,1690229447504.437177da113d7ca44d1d74b0ae731139.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229448243"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690229448243"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690229448243"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690229448243"}]},"ts":"1690229448243"} 2023-07-24 20:10:48,243 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for Group_testCreateMultiRegion,\x00bdfh,1690229447504.98abe1672c7585d97343038205bd5450. 2023-07-24 20:10:48,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened Group_testCreateMultiRegion,\x00bdfh,1690229447504.98abe1672c7585d97343038205bd5450. 2023-07-24 20:10:48,244 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=98abe1672c7585d97343038205bd5450, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:48,244 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"Group_testCreateMultiRegion,\\x00bdfh,1690229447504.98abe1672c7585d97343038205bd5450.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229448244"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690229448244"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690229448244"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690229448244"}]},"ts":"1690229448244"} 2023-07-24 20:10:48,245 INFO [StoreOpener-7f557baf61410e30c82ed736835597ae-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f of region 7f557baf61410e30c82ed736835597ae 2023-07-24 20:10:48,248 DEBUG [StoreOpener-7f557baf61410e30c82ed736835597ae-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/7f557baf61410e30c82ed736835597ae/f 2023-07-24 20:10:48,248 DEBUG [StoreOpener-7f557baf61410e30c82ed736835597ae-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/7f557baf61410e30c82ed736835597ae/f 2023-07-24 20:10:48,248 INFO [StoreOpener-7f557baf61410e30c82ed736835597ae-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7f557baf61410e30c82ed736835597ae columnFamilyName f 2023-07-24 20:10:48,249 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=33 2023-07-24 20:10:48,249 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=33, state=SUCCESS; OpenRegionProcedure 437177da113d7ca44d1d74b0ae731139, server=jenkins-hbase4.apache.org,34483,1690229439858 in 285 msec 2023-07-24 20:10:48,250 INFO [StoreOpener-7f557baf61410e30c82ed736835597ae-1] regionserver.HStore(310): Store=7f557baf61410e30c82ed736835597ae/f, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:10:48,251 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=40, resume processing ppid=29 2023-07-24 20:10:48,251 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=40, ppid=29, state=SUCCESS; OpenRegionProcedure 98abe1672c7585d97343038205bd5450, server=jenkins-hbase4.apache.org,34917,1690229443609 in 291 msec 2023-07-24 20:10:48,251 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/7f557baf61410e30c82ed736835597ae 2023-07-24 20:10:48,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/7f557baf61410e30c82ed736835597ae 2023-07-24 20:10:48,252 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=24, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=437177da113d7ca44d1d74b0ae731139, ASSIGN in 471 msec 2023-07-24 20:10:48,253 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=24, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=98abe1672c7585d97343038205bd5450, ASSIGN in 473 msec 2023-07-24 20:10:48,256 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 7f557baf61410e30c82ed736835597ae 2023-07-24 20:10:48,258 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/7f557baf61410e30c82ed736835597ae/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 20:10:48,259 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 7f557baf61410e30c82ed736835597ae; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=11847554560, jitterRate=0.10338950157165527}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 20:10:48,259 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 7f557baf61410e30c82ed736835597ae: 2023-07-24 20:10:48,260 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1690229447504.7f557baf61410e30c82ed736835597ae., pid=44, masterSystemTime=1690229448113 2023-07-24 20:10:48,262 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1690229447504.7f557baf61410e30c82ed736835597ae. 2023-07-24 20:10:48,262 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1690229447504.7f557baf61410e30c82ed736835597ae. 2023-07-24 20:10:48,263 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=7f557baf61410e30c82ed736835597ae, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:48,263 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"Group_testCreateMultiRegion,\\x00\\x02\\x04\\x06\\x08,1690229447504.7f557baf61410e30c82ed736835597ae.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229448262"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690229448262"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690229448262"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690229448262"}]},"ts":"1690229448262"} 2023-07-24 20:10:48,267 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=44, resume processing ppid=26 2023-07-24 20:10:48,267 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=44, ppid=26, state=SUCCESS; OpenRegionProcedure 7f557baf61410e30c82ed736835597ae, server=jenkins-hbase4.apache.org,34483,1690229439858 in 303 msec 2023-07-24 20:10:48,270 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=24 2023-07-24 20:10:48,270 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=24, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=7f557baf61410e30c82ed736835597ae, ASSIGN in 489 msec 2023-07-24 20:10:48,271 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=24, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=Group_testCreateMultiRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-24 20:10:48,272 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCreateMultiRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690229448272"}]},"ts":"1690229448272"} 2023-07-24 20:10:48,274 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCreateMultiRegion, state=ENABLED in hbase:meta 2023-07-24 20:10:48,278 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=24, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=Group_testCreateMultiRegion execute state=CREATE_TABLE_POST_OPERATION 2023-07-24 20:10:48,281 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; CreateTableProcedure table=Group_testCreateMultiRegion in 773 msec 2023-07-24 20:10:48,629 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=24 2023-07-24 20:10:48,630 INFO [Listener at localhost/44145] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:Group_testCreateMultiRegion, procId: 24 completed 2023-07-24 20:10:48,630 DEBUG [Listener at localhost/44145] hbase.HBaseTestingUtility(3430): Waiting until all regions of table Group_testCreateMultiRegion get assigned. Timeout = 60000ms 2023-07-24 20:10:48,631 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:10:48,634 WARN [RPCClient-NioEventLoopGroup-6-1] ipc.NettyRpcConnection$2(294): Exception encountered while connecting to the server jenkins-hbase4.apache.org/172.31.14.131:34371 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:34371 Caused by: java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:716) at org.apache.hbase.thirdparty.io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:337) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:334) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:776) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-07-24 20:10:48,634 DEBUG [RPCClient-NioEventLoopGroup-6-1] ipc.FailedServers(52): Added failed server with address jenkins-hbase4.apache.org/172.31.14.131:34371 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:34371 2023-07-24 20:10:48,737 DEBUG [hconnection-0x4636fd6e-shared-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-24 20:10:48,740 INFO [RS-EventLoopGroup-7-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51920, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-24 20:10:48,758 INFO [Listener at localhost/44145] hbase.HBaseTestingUtility(3484): All regions for table Group_testCreateMultiRegion assigned to meta. Checking AM states. 2023-07-24 20:10:48,759 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:10:48,759 INFO [Listener at localhost/44145] hbase.HBaseTestingUtility(3504): All regions for table Group_testCreateMultiRegion assigned. 2023-07-24 20:10:48,761 INFO [Listener at localhost/44145] client.HBaseAdmin$15(890): Started disable of Group_testCreateMultiRegion 2023-07-24 20:10:48,762 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable Group_testCreateMultiRegion 2023-07-24 20:10:48,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] procedure2.ProcedureExecutor(1029): Stored pid=45, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=Group_testCreateMultiRegion 2023-07-24 20:10:48,766 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=45 2023-07-24 20:10:48,770 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCreateMultiRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690229448770"}]},"ts":"1690229448770"} 2023-07-24 20:10:48,772 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCreateMultiRegion, state=DISABLING in hbase:meta 2023-07-24 20:10:48,774 INFO [PEWorker-4] procedure.DisableTableProcedure(293): Set Group_testCreateMultiRegion to state=DISABLING 2023-07-24 20:10:48,779 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=7f557baf61410e30c82ed736835597ae, UNASSIGN}, {pid=47, ppid=45, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=486c039d584a40408ca57b59a9685da6, UNASSIGN}, {pid=48, ppid=45, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=83406e51fcf8abcdba40d7c315de5777, UNASSIGN}, {pid=49, ppid=45, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=98abe1672c7585d97343038205bd5450, UNASSIGN}, {pid=50, ppid=45, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=8a701529ae84652c9c029ed9bd063968, UNASSIGN}, {pid=51, ppid=45, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=2df6ec0be254267ea2d62569ac1c609d, UNASSIGN}, {pid=52, ppid=45, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=30fd729e25b9ba973f8749e1094e1710, UNASSIGN}, {pid=53, ppid=45, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=437177da113d7ca44d1d74b0ae731139, UNASSIGN}, {pid=54, ppid=45, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=b3a80fc52bfa48de445ea8e679dbc14a, UNASSIGN}, {pid=55, ppid=45, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=f55a12abc8e0bcbed2f6e2d3644e4a53, UNASSIGN}] 2023-07-24 20:10:48,780 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=45, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=b3a80fc52bfa48de445ea8e679dbc14a, UNASSIGN 2023-07-24 20:10:48,781 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=55, ppid=45, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=f55a12abc8e0bcbed2f6e2d3644e4a53, UNASSIGN 2023-07-24 20:10:48,781 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=53, ppid=45, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=437177da113d7ca44d1d74b0ae731139, UNASSIGN 2023-07-24 20:10:48,782 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=52, ppid=45, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=30fd729e25b9ba973f8749e1094e1710, UNASSIGN 2023-07-24 20:10:48,782 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=51, ppid=45, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=2df6ec0be254267ea2d62569ac1c609d, UNASSIGN 2023-07-24 20:10:48,783 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=b3a80fc52bfa48de445ea8e679dbc14a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:48,784 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=55 updating hbase:meta row=f55a12abc8e0bcbed2f6e2d3644e4a53, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:48,784 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x01\\x03\\x05\\x07\\x09,1690229447504.b3a80fc52bfa48de445ea8e679dbc14a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1690229448783"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229448783"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229448783"}]},"ts":"1690229448783"} 2023-07-24 20:10:48,784 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=53 updating hbase:meta row=437177da113d7ca44d1d74b0ae731139, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:48,784 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,,1690229447504.f55a12abc8e0bcbed2f6e2d3644e4a53.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1690229448784"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229448784"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229448784"}]},"ts":"1690229448784"} 2023-07-24 20:10:48,784 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=2df6ec0be254267ea2d62569ac1c609d, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:48,784 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x00\\xE2\\xE4\\xE6\\xE8,1690229447504.437177da113d7ca44d1d74b0ae731139.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229448784"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229448784"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229448784"}]},"ts":"1690229448784"} 2023-07-24 20:10:48,784 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=52 updating hbase:meta row=30fd729e25b9ba973f8749e1094e1710, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:48,784 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x00\\xA2\\xA4\\xA6\\xA8,1690229447504.2df6ec0be254267ea2d62569ac1c609d.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229448784"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229448784"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229448784"}]},"ts":"1690229448784"} 2023-07-24 20:10:48,784 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x00\\xC2\\xC4\\xC6\\xC8,1690229447504.30fd729e25b9ba973f8749e1094e1710.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229448784"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229448784"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229448784"}]},"ts":"1690229448784"} 2023-07-24 20:10:48,787 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=56, ppid=54, state=RUNNABLE; CloseRegionProcedure b3a80fc52bfa48de445ea8e679dbc14a, server=jenkins-hbase4.apache.org,34917,1690229443609}] 2023-07-24 20:10:48,788 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=55, state=RUNNABLE; CloseRegionProcedure f55a12abc8e0bcbed2f6e2d3644e4a53, server=jenkins-hbase4.apache.org,34917,1690229443609}] 2023-07-24 20:10:48,790 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=53, state=RUNNABLE; CloseRegionProcedure 437177da113d7ca44d1d74b0ae731139, server=jenkins-hbase4.apache.org,34483,1690229439858}] 2023-07-24 20:10:48,791 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=51, state=RUNNABLE; CloseRegionProcedure 2df6ec0be254267ea2d62569ac1c609d, server=jenkins-hbase4.apache.org,39791,1690229440084}] 2023-07-24 20:10:48,792 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=60, ppid=52, state=RUNNABLE; CloseRegionProcedure 30fd729e25b9ba973f8749e1094e1710, server=jenkins-hbase4.apache.org,45209,1690229445078}] 2023-07-24 20:10:48,793 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=45, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=8a701529ae84652c9c029ed9bd063968, UNASSIGN 2023-07-24 20:10:48,795 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=8a701529ae84652c9c029ed9bd063968, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:48,795 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x00\\x82\\x84\\x86\\x88,1690229447504.8a701529ae84652c9c029ed9bd063968.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229448795"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229448795"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229448795"}]},"ts":"1690229448795"} 2023-07-24 20:10:48,796 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=49, ppid=45, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=98abe1672c7585d97343038205bd5450, UNASSIGN 2023-07-24 20:10:48,797 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=48, ppid=45, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=83406e51fcf8abcdba40d7c315de5777, UNASSIGN 2023-07-24 20:10:48,797 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, ppid=45, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=486c039d584a40408ca57b59a9685da6, UNASSIGN 2023-07-24 20:10:48,797 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=49 updating hbase:meta row=98abe1672c7585d97343038205bd5450, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:48,797 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x00bdfh,1690229447504.98abe1672c7585d97343038205bd5450.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229448797"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229448797"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229448797"}]},"ts":"1690229448797"} 2023-07-24 20:10:48,798 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=50, state=RUNNABLE; CloseRegionProcedure 8a701529ae84652c9c029ed9bd063968, server=jenkins-hbase4.apache.org,34483,1690229439858}] 2023-07-24 20:10:48,798 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=48 updating hbase:meta row=83406e51fcf8abcdba40d7c315de5777, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:48,798 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x00BDFH,1690229447504.83406e51fcf8abcdba40d7c315de5777.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229448798"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229448798"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229448798"}]},"ts":"1690229448798"} 2023-07-24 20:10:48,799 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=46, ppid=45, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=7f557baf61410e30c82ed736835597ae, UNASSIGN 2023-07-24 20:10:48,799 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=486c039d584a40408ca57b59a9685da6, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:48,799 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x00\"$\u0026(,1690229447504.486c039d584a40408ca57b59a9685da6.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229448799"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229448799"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229448799"}]},"ts":"1690229448799"} 2023-07-24 20:10:48,800 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=49, state=RUNNABLE; CloseRegionProcedure 98abe1672c7585d97343038205bd5450, server=jenkins-hbase4.apache.org,34917,1690229443609}] 2023-07-24 20:10:48,801 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=63, ppid=48, state=RUNNABLE; CloseRegionProcedure 83406e51fcf8abcdba40d7c315de5777, server=jenkins-hbase4.apache.org,45209,1690229445078}] 2023-07-24 20:10:48,802 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=46 updating hbase:meta row=7f557baf61410e30c82ed736835597ae, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:48,802 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateMultiRegion,\\x00\\x02\\x04\\x06\\x08,1690229447504.7f557baf61410e30c82ed736835597ae.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229448801"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229448801"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229448801"}]},"ts":"1690229448801"} 2023-07-24 20:10:48,802 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=47, state=RUNNABLE; CloseRegionProcedure 486c039d584a40408ca57b59a9685da6, server=jenkins-hbase4.apache.org,39791,1690229440084}] 2023-07-24 20:10:48,804 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=65, ppid=46, state=RUNNABLE; CloseRegionProcedure 7f557baf61410e30c82ed736835597ae, server=jenkins-hbase4.apache.org,34483,1690229439858}] 2023-07-24 20:10:48,868 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=45 2023-07-24 20:10:48,941 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 98abe1672c7585d97343038205bd5450 2023-07-24 20:10:48,942 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 98abe1672c7585d97343038205bd5450, disabling compactions & flushes 2023-07-24 20:10:48,942 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x00bdfh,1690229447504.98abe1672c7585d97343038205bd5450. 2023-07-24 20:10:48,942 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x00bdfh,1690229447504.98abe1672c7585d97343038205bd5450. 2023-07-24 20:10:48,942 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x00bdfh,1690229447504.98abe1672c7585d97343038205bd5450. after waiting 0 ms 2023-07-24 20:10:48,942 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x00bdfh,1690229447504.98abe1672c7585d97343038205bd5450. 2023-07-24 20:10:48,944 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 8a701529ae84652c9c029ed9bd063968 2023-07-24 20:10:48,945 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 8a701529ae84652c9c029ed9bd063968, disabling compactions & flushes 2023-07-24 20:10:48,945 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1690229447504.8a701529ae84652c9c029ed9bd063968. 2023-07-24 20:10:48,945 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1690229447504.8a701529ae84652c9c029ed9bd063968. 2023-07-24 20:10:48,945 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1690229447504.8a701529ae84652c9c029ed9bd063968. after waiting 0 ms 2023-07-24 20:10:48,945 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1690229447504.8a701529ae84652c9c029ed9bd063968. 2023-07-24 20:10:48,945 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 83406e51fcf8abcdba40d7c315de5777 2023-07-24 20:10:48,947 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 83406e51fcf8abcdba40d7c315de5777, disabling compactions & flushes 2023-07-24 20:10:48,947 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x00BDFH,1690229447504.83406e51fcf8abcdba40d7c315de5777. 2023-07-24 20:10:48,947 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x00BDFH,1690229447504.83406e51fcf8abcdba40d7c315de5777. 2023-07-24 20:10:48,947 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x00BDFH,1690229447504.83406e51fcf8abcdba40d7c315de5777. after waiting 0 ms 2023-07-24 20:10:48,947 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x00BDFH,1690229447504.83406e51fcf8abcdba40d7c315de5777. 2023-07-24 20:10:48,947 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 486c039d584a40408ca57b59a9685da6 2023-07-24 20:10:48,949 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 486c039d584a40408ca57b59a9685da6, disabling compactions & flushes 2023-07-24 20:10:48,949 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x00"$&(,1690229447504.486c039d584a40408ca57b59a9685da6. 2023-07-24 20:10:48,949 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x00"$&(,1690229447504.486c039d584a40408ca57b59a9685da6. 2023-07-24 20:10:48,949 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x00"$&(,1690229447504.486c039d584a40408ca57b59a9685da6. after waiting 0 ms 2023-07-24 20:10:48,949 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x00"$&(,1690229447504.486c039d584a40408ca57b59a9685da6. 2023-07-24 20:10:48,965 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/8a701529ae84652c9c029ed9bd063968/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-24 20:10:48,965 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/98abe1672c7585d97343038205bd5450/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-24 20:10:48,966 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1690229447504.8a701529ae84652c9c029ed9bd063968. 2023-07-24 20:10:48,966 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x00bdfh,1690229447504.98abe1672c7585d97343038205bd5450. 2023-07-24 20:10:48,966 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 98abe1672c7585d97343038205bd5450: 2023-07-24 20:10:48,966 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 8a701529ae84652c9c029ed9bd063968: 2023-07-24 20:10:48,966 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/486c039d584a40408ca57b59a9685da6/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-24 20:10:48,967 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/83406e51fcf8abcdba40d7c315de5777/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-24 20:10:48,967 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x00"$&(,1690229447504.486c039d584a40408ca57b59a9685da6. 2023-07-24 20:10:48,968 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 486c039d584a40408ca57b59a9685da6: 2023-07-24 20:10:48,969 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x00BDFH,1690229447504.83406e51fcf8abcdba40d7c315de5777. 2023-07-24 20:10:48,969 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 83406e51fcf8abcdba40d7c315de5777: 2023-07-24 20:10:48,970 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 98abe1672c7585d97343038205bd5450 2023-07-24 20:10:48,970 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close b3a80fc52bfa48de445ea8e679dbc14a 2023-07-24 20:10:48,971 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b3a80fc52bfa48de445ea8e679dbc14a, disabling compactions & flushes 2023-07-24 20:10:48,972 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1690229447504.b3a80fc52bfa48de445ea8e679dbc14a. 2023-07-24 20:10:48,972 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1690229447504.b3a80fc52bfa48de445ea8e679dbc14a. 2023-07-24 20:10:48,972 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1690229447504.b3a80fc52bfa48de445ea8e679dbc14a. after waiting 0 ms 2023-07-24 20:10:48,972 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1690229447504.b3a80fc52bfa48de445ea8e679dbc14a. 2023-07-24 20:10:48,977 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=49 updating hbase:meta row=98abe1672c7585d97343038205bd5450, regionState=CLOSED 2023-07-24 20:10:48,977 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x00bdfh,1690229447504.98abe1672c7585d97343038205bd5450.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229448977"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229448977"}]},"ts":"1690229448977"} 2023-07-24 20:10:48,979 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=8a701529ae84652c9c029ed9bd063968, regionState=CLOSED 2023-07-24 20:10:48,979 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x00\\x82\\x84\\x86\\x88,1690229447504.8a701529ae84652c9c029ed9bd063968.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229448978"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229448978"}]},"ts":"1690229448978"} 2023-07-24 20:10:48,979 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 486c039d584a40408ca57b59a9685da6 2023-07-24 20:10:48,980 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 2df6ec0be254267ea2d62569ac1c609d 2023-07-24 20:10:48,981 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=486c039d584a40408ca57b59a9685da6, regionState=CLOSED 2023-07-24 20:10:48,981 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x00\"$\u0026(,1690229447504.486c039d584a40408ca57b59a9685da6.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229448981"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229448981"}]},"ts":"1690229448981"} 2023-07-24 20:10:48,981 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 83406e51fcf8abcdba40d7c315de5777 2023-07-24 20:10:48,981 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 30fd729e25b9ba973f8749e1094e1710 2023-07-24 20:10:48,982 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 8a701529ae84652c9c029ed9bd063968 2023-07-24 20:10:48,982 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 437177da113d7ca44d1d74b0ae731139 2023-07-24 20:10:48,983 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=48 updating hbase:meta row=83406e51fcf8abcdba40d7c315de5777, regionState=CLOSED 2023-07-24 20:10:48,984 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x00BDFH,1690229447504.83406e51fcf8abcdba40d7c315de5777.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229448983"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229448983"}]},"ts":"1690229448983"} 2023-07-24 20:10:48,987 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2df6ec0be254267ea2d62569ac1c609d, disabling compactions & flushes 2023-07-24 20:10:48,987 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1690229447504.2df6ec0be254267ea2d62569ac1c609d. 2023-07-24 20:10:48,987 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1690229447504.2df6ec0be254267ea2d62569ac1c609d. 2023-07-24 20:10:48,987 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1690229447504.2df6ec0be254267ea2d62569ac1c609d. after waiting 0 ms 2023-07-24 20:10:48,987 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1690229447504.2df6ec0be254267ea2d62569ac1c609d. 2023-07-24 20:10:48,987 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 30fd729e25b9ba973f8749e1094e1710, disabling compactions & flushes 2023-07-24 20:10:48,987 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 437177da113d7ca44d1d74b0ae731139, disabling compactions & flushes 2023-07-24 20:10:48,987 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1690229447504.30fd729e25b9ba973f8749e1094e1710. 2023-07-24 20:10:48,987 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1690229447504.437177da113d7ca44d1d74b0ae731139. 2023-07-24 20:10:48,987 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1690229447504.30fd729e25b9ba973f8749e1094e1710. 2023-07-24 20:10:48,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1690229447504.437177da113d7ca44d1d74b0ae731139. 2023-07-24 20:10:48,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1690229447504.437177da113d7ca44d1d74b0ae731139. after waiting 0 ms 2023-07-24 20:10:48,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1690229447504.437177da113d7ca44d1d74b0ae731139. 2023-07-24 20:10:48,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1690229447504.30fd729e25b9ba973f8749e1094e1710. after waiting 0 ms 2023-07-24 20:10:48,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1690229447504.30fd729e25b9ba973f8749e1094e1710. 2023-07-24 20:10:48,992 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=49 2023-07-24 20:10:48,992 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=49, state=SUCCESS; CloseRegionProcedure 98abe1672c7585d97343038205bd5450, server=jenkins-hbase4.apache.org,34917,1690229443609 in 181 msec 2023-07-24 20:10:48,994 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=50 2023-07-24 20:10:48,994 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=50, state=SUCCESS; CloseRegionProcedure 8a701529ae84652c9c029ed9bd063968, server=jenkins-hbase4.apache.org,34483,1690229439858 in 185 msec 2023-07-24 20:10:48,996 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=64, resume processing ppid=47 2023-07-24 20:10:48,996 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=47, state=SUCCESS; CloseRegionProcedure 486c039d584a40408ca57b59a9685da6, server=jenkins-hbase4.apache.org,39791,1690229440084 in 187 msec 2023-07-24 20:10:48,996 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=63, resume processing ppid=48 2023-07-24 20:10:48,997 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, ppid=48, state=SUCCESS; CloseRegionProcedure 83406e51fcf8abcdba40d7c315de5777, server=jenkins-hbase4.apache.org,45209,1690229445078 in 188 msec 2023-07-24 20:10:48,997 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=49, ppid=45, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=98abe1672c7585d97343038205bd5450, UNASSIGN in 217 msec 2023-07-24 20:10:48,997 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=45, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=8a701529ae84652c9c029ed9bd063968, UNASSIGN in 216 msec 2023-07-24 20:10:48,999 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=47, ppid=45, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=486c039d584a40408ca57b59a9685da6, UNASSIGN in 221 msec 2023-07-24 20:10:48,999 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=45, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=83406e51fcf8abcdba40d7c315de5777, UNASSIGN in 222 msec 2023-07-24 20:10:49,011 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/b3a80fc52bfa48de445ea8e679dbc14a/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-24 20:10:49,012 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1690229447504.b3a80fc52bfa48de445ea8e679dbc14a. 2023-07-24 20:10:49,012 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b3a80fc52bfa48de445ea8e679dbc14a: 2023-07-24 20:10:49,013 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/2df6ec0be254267ea2d62569ac1c609d/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-24 20:10:49,016 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1690229447504.2df6ec0be254267ea2d62569ac1c609d. 2023-07-24 20:10:49,016 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2df6ec0be254267ea2d62569ac1c609d: 2023-07-24 20:10:49,019 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed b3a80fc52bfa48de445ea8e679dbc14a 2023-07-24 20:10:49,019 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close f55a12abc8e0bcbed2f6e2d3644e4a53 2023-07-24 20:10:49,020 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=b3a80fc52bfa48de445ea8e679dbc14a, regionState=CLOSED 2023-07-24 20:10:49,020 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x01\\x03\\x05\\x07\\x09,1690229447504.b3a80fc52bfa48de445ea8e679dbc14a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1690229449019"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229449019"}]},"ts":"1690229449019"} 2023-07-24 20:10:49,022 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=2df6ec0be254267ea2d62569ac1c609d, regionState=CLOSED 2023-07-24 20:10:49,022 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x00\\xA2\\xA4\\xA6\\xA8,1690229447504.2df6ec0be254267ea2d62569ac1c609d.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229449022"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229449022"}]},"ts":"1690229449022"} 2023-07-24 20:10:49,022 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 2df6ec0be254267ea2d62569ac1c609d 2023-07-24 20:10:49,026 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=56, resume processing ppid=54 2023-07-24 20:10:49,027 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=56, ppid=54, state=SUCCESS; CloseRegionProcedure b3a80fc52bfa48de445ea8e679dbc14a, server=jenkins-hbase4.apache.org,34917,1690229443609 in 235 msec 2023-07-24 20:10:49,031 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing f55a12abc8e0bcbed2f6e2d3644e4a53, disabling compactions & flushes 2023-07-24 20:10:49,031 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,,1690229447504.f55a12abc8e0bcbed2f6e2d3644e4a53. 2023-07-24 20:10:49,031 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,,1690229447504.f55a12abc8e0bcbed2f6e2d3644e4a53. 2023-07-24 20:10:49,031 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,,1690229447504.f55a12abc8e0bcbed2f6e2d3644e4a53. after waiting 0 ms 2023-07-24 20:10:49,031 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,,1690229447504.f55a12abc8e0bcbed2f6e2d3644e4a53. 2023-07-24 20:10:49,032 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/437177da113d7ca44d1d74b0ae731139/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-24 20:10:49,033 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/30fd729e25b9ba973f8749e1094e1710/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-24 20:10:49,040 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1690229447504.437177da113d7ca44d1d74b0ae731139. 2023-07-24 20:10:49,040 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 437177da113d7ca44d1d74b0ae731139: 2023-07-24 20:10:49,042 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1690229447504.30fd729e25b9ba973f8749e1094e1710. 2023-07-24 20:10:49,042 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 30fd729e25b9ba973f8749e1094e1710: 2023-07-24 20:10:49,051 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=59, resume processing ppid=51 2023-07-24 20:10:49,051 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=45, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=b3a80fc52bfa48de445ea8e679dbc14a, UNASSIGN in 249 msec 2023-07-24 20:10:49,051 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=51, state=SUCCESS; CloseRegionProcedure 2df6ec0be254267ea2d62569ac1c609d, server=jenkins-hbase4.apache.org,39791,1690229440084 in 234 msec 2023-07-24 20:10:49,052 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 437177da113d7ca44d1d74b0ae731139 2023-07-24 20:10:49,052 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 7f557baf61410e30c82ed736835597ae 2023-07-24 20:10:49,053 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=53 updating hbase:meta row=437177da113d7ca44d1d74b0ae731139, regionState=CLOSED 2023-07-24 20:10:49,053 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x00\\xE2\\xE4\\xE6\\xE8,1690229447504.437177da113d7ca44d1d74b0ae731139.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229449053"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229449053"}]},"ts":"1690229449053"} 2023-07-24 20:10:49,053 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 30fd729e25b9ba973f8749e1094e1710 2023-07-24 20:10:49,054 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=45, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=2df6ec0be254267ea2d62569ac1c609d, UNASSIGN in 273 msec 2023-07-24 20:10:49,054 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=52 updating hbase:meta row=30fd729e25b9ba973f8749e1094e1710, regionState=CLOSED 2023-07-24 20:10:49,054 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x00\\xC2\\xC4\\xC6\\xC8,1690229447504.30fd729e25b9ba973f8749e1094e1710.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229449054"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229449054"}]},"ts":"1690229449054"} 2023-07-24 20:10:49,059 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=52 2023-07-24 20:10:49,059 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=52, state=SUCCESS; CloseRegionProcedure 30fd729e25b9ba973f8749e1094e1710, server=jenkins-hbase4.apache.org,45209,1690229445078 in 265 msec 2023-07-24 20:10:49,062 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 7f557baf61410e30c82ed736835597ae, disabling compactions & flushes 2023-07-24 20:10:49,062 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1690229447504.7f557baf61410e30c82ed736835597ae. 2023-07-24 20:10:49,062 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1690229447504.7f557baf61410e30c82ed736835597ae. 2023-07-24 20:10:49,062 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1690229447504.7f557baf61410e30c82ed736835597ae. after waiting 0 ms 2023-07-24 20:10:49,062 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1690229447504.7f557baf61410e30c82ed736835597ae. 2023-07-24 20:10:49,063 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/f55a12abc8e0bcbed2f6e2d3644e4a53/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-24 20:10:49,064 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=52, ppid=45, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=30fd729e25b9ba973f8749e1094e1710, UNASSIGN in 281 msec 2023-07-24 20:10:49,067 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=53 2023-07-24 20:10:49,067 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,,1690229447504.f55a12abc8e0bcbed2f6e2d3644e4a53. 2023-07-24 20:10:49,068 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=53, state=SUCCESS; CloseRegionProcedure 437177da113d7ca44d1d74b0ae731139, server=jenkins-hbase4.apache.org,34483,1690229439858 in 266 msec 2023-07-24 20:10:49,068 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for f55a12abc8e0bcbed2f6e2d3644e4a53: 2023-07-24 20:10:49,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=45 2023-07-24 20:10:49,070 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed f55a12abc8e0bcbed2f6e2d3644e4a53 2023-07-24 20:10:49,071 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=53, ppid=45, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=437177da113d7ca44d1d74b0ae731139, UNASSIGN in 290 msec 2023-07-24 20:10:49,071 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=55 updating hbase:meta row=f55a12abc8e0bcbed2f6e2d3644e4a53, regionState=CLOSED 2023-07-24 20:10:49,071 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,,1690229447504.f55a12abc8e0bcbed2f6e2d3644e4a53.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1690229449071"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229449071"}]},"ts":"1690229449071"} 2023-07-24 20:10:49,075 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=55 2023-07-24 20:10:49,075 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=55, state=SUCCESS; CloseRegionProcedure f55a12abc8e0bcbed2f6e2d3644e4a53, server=jenkins-hbase4.apache.org,34917,1690229443609 in 285 msec 2023-07-24 20:10:49,077 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=45, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=f55a12abc8e0bcbed2f6e2d3644e4a53, UNASSIGN in 297 msec 2023-07-24 20:10:49,083 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateMultiRegion/7f557baf61410e30c82ed736835597ae/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-24 20:10:49,084 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1690229447504.7f557baf61410e30c82ed736835597ae. 2023-07-24 20:10:49,085 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 7f557baf61410e30c82ed736835597ae: 2023-07-24 20:10:49,087 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 7f557baf61410e30c82ed736835597ae 2023-07-24 20:10:49,088 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=46 updating hbase:meta row=7f557baf61410e30c82ed736835597ae, regionState=CLOSED 2023-07-24 20:10:49,088 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"Group_testCreateMultiRegion,\\x00\\x02\\x04\\x06\\x08,1690229447504.7f557baf61410e30c82ed736835597ae.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229449088"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229449088"}]},"ts":"1690229449088"} 2023-07-24 20:10:49,092 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=65, resume processing ppid=46 2023-07-24 20:10:49,093 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=46, state=SUCCESS; CloseRegionProcedure 7f557baf61410e30c82ed736835597ae, server=jenkins-hbase4.apache.org,34483,1690229439858 in 286 msec 2023-07-24 20:10:49,095 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-07-24 20:10:49,095 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateMultiRegion, region=7f557baf61410e30c82ed736835597ae, UNASSIGN in 317 msec 2023-07-24 20:10:49,096 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCreateMultiRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690229449096"}]},"ts":"1690229449096"} 2023-07-24 20:10:49,098 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCreateMultiRegion, state=DISABLED in hbase:meta 2023-07-24 20:10:49,100 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set Group_testCreateMultiRegion to state=DISABLED 2023-07-24 20:10:49,103 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, state=SUCCESS; DisableTableProcedure table=Group_testCreateMultiRegion in 339 msec 2023-07-24 20:10:49,371 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=45 2023-07-24 20:10:49,371 INFO [Listener at localhost/44145] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:Group_testCreateMultiRegion, procId: 45 completed 2023-07-24 20:10:49,372 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete Group_testCreateMultiRegion 2023-07-24 20:10:49,374 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] procedure2.ProcedureExecutor(1029): Stored pid=66, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=Group_testCreateMultiRegion 2023-07-24 20:10:49,376 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=66, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=Group_testCreateMultiRegion 2023-07-24 20:10:49,376 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint(577): Removing deleted table 'Group_testCreateMultiRegion' from rsgroup 'default' 2023-07-24 20:10:49,378 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=66, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=Group_testCreateMultiRegion 2023-07-24 20:10:49,380 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:49,380 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:49,381 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-24 20:10:49,384 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=66 2023-07-24 20:10:49,397 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/7f557baf61410e30c82ed736835597ae 2023-07-24 20:10:49,398 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/8a701529ae84652c9c029ed9bd063968 2023-07-24 20:10:49,398 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/437177da113d7ca44d1d74b0ae731139 2023-07-24 20:10:49,398 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/30fd729e25b9ba973f8749e1094e1710 2023-07-24 20:10:49,398 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/2df6ec0be254267ea2d62569ac1c609d 2023-07-24 20:10:49,397 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/486c039d584a40408ca57b59a9685da6 2023-07-24 20:10:49,398 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/98abe1672c7585d97343038205bd5450 2023-07-24 20:10:49,398 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/83406e51fcf8abcdba40d7c315de5777 2023-07-24 20:10:49,403 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/486c039d584a40408ca57b59a9685da6/f, FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/486c039d584a40408ca57b59a9685da6/recovered.edits] 2023-07-24 20:10:49,404 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/437177da113d7ca44d1d74b0ae731139/f, FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/437177da113d7ca44d1d74b0ae731139/recovered.edits] 2023-07-24 20:10:49,404 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/83406e51fcf8abcdba40d7c315de5777/f, FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/83406e51fcf8abcdba40d7c315de5777/recovered.edits] 2023-07-24 20:10:49,406 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/8a701529ae84652c9c029ed9bd063968/f, FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/8a701529ae84652c9c029ed9bd063968/recovered.edits] 2023-07-24 20:10:49,407 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/7f557baf61410e30c82ed736835597ae/f, FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/7f557baf61410e30c82ed736835597ae/recovered.edits] 2023-07-24 20:10:49,407 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/98abe1672c7585d97343038205bd5450/f, FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/98abe1672c7585d97343038205bd5450/recovered.edits] 2023-07-24 20:10:49,407 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/2df6ec0be254267ea2d62569ac1c609d/f, FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/2df6ec0be254267ea2d62569ac1c609d/recovered.edits] 2023-07-24 20:10:49,408 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/30fd729e25b9ba973f8749e1094e1710/f, FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/30fd729e25b9ba973f8749e1094e1710/recovered.edits] 2023-07-24 20:10:49,426 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/7f557baf61410e30c82ed736835597ae/recovered.edits/4.seqid to hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/archive/data/default/Group_testCreateMultiRegion/7f557baf61410e30c82ed736835597ae/recovered.edits/4.seqid 2023-07-24 20:10:49,428 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/7f557baf61410e30c82ed736835597ae 2023-07-24 20:10:49,428 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/b3a80fc52bfa48de445ea8e679dbc14a 2023-07-24 20:10:49,429 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/486c039d584a40408ca57b59a9685da6/recovered.edits/4.seqid to hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/archive/data/default/Group_testCreateMultiRegion/486c039d584a40408ca57b59a9685da6/recovered.edits/4.seqid 2023-07-24 20:10:49,430 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/437177da113d7ca44d1d74b0ae731139/recovered.edits/4.seqid to hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/archive/data/default/Group_testCreateMultiRegion/437177da113d7ca44d1d74b0ae731139/recovered.edits/4.seqid 2023-07-24 20:10:49,431 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/2df6ec0be254267ea2d62569ac1c609d/recovered.edits/4.seqid to hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/archive/data/default/Group_testCreateMultiRegion/2df6ec0be254267ea2d62569ac1c609d/recovered.edits/4.seqid 2023-07-24 20:10:49,431 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/486c039d584a40408ca57b59a9685da6 2023-07-24 20:10:49,431 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/f55a12abc8e0bcbed2f6e2d3644e4a53 2023-07-24 20:10:49,432 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/437177da113d7ca44d1d74b0ae731139 2023-07-24 20:10:49,432 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/30fd729e25b9ba973f8749e1094e1710/recovered.edits/4.seqid to hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/archive/data/default/Group_testCreateMultiRegion/30fd729e25b9ba973f8749e1094e1710/recovered.edits/4.seqid 2023-07-24 20:10:49,432 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/83406e51fcf8abcdba40d7c315de5777/recovered.edits/4.seqid to hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/archive/data/default/Group_testCreateMultiRegion/83406e51fcf8abcdba40d7c315de5777/recovered.edits/4.seqid 2023-07-24 20:10:49,433 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/8a701529ae84652c9c029ed9bd063968/recovered.edits/4.seqid to hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/archive/data/default/Group_testCreateMultiRegion/8a701529ae84652c9c029ed9bd063968/recovered.edits/4.seqid 2023-07-24 20:10:49,433 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/2df6ec0be254267ea2d62569ac1c609d 2023-07-24 20:10:49,434 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/30fd729e25b9ba973f8749e1094e1710 2023-07-24 20:10:49,435 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/83406e51fcf8abcdba40d7c315de5777 2023-07-24 20:10:49,435 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/8a701529ae84652c9c029ed9bd063968 2023-07-24 20:10:49,435 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/b3a80fc52bfa48de445ea8e679dbc14a/f, FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/b3a80fc52bfa48de445ea8e679dbc14a/recovered.edits] 2023-07-24 20:10:49,436 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/98abe1672c7585d97343038205bd5450/recovered.edits/4.seqid to hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/archive/data/default/Group_testCreateMultiRegion/98abe1672c7585d97343038205bd5450/recovered.edits/4.seqid 2023-07-24 20:10:49,436 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/f55a12abc8e0bcbed2f6e2d3644e4a53/f, FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/f55a12abc8e0bcbed2f6e2d3644e4a53/recovered.edits] 2023-07-24 20:10:49,436 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/98abe1672c7585d97343038205bd5450 2023-07-24 20:10:49,443 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/b3a80fc52bfa48de445ea8e679dbc14a/recovered.edits/4.seqid to hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/archive/data/default/Group_testCreateMultiRegion/b3a80fc52bfa48de445ea8e679dbc14a/recovered.edits/4.seqid 2023-07-24 20:10:49,444 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/f55a12abc8e0bcbed2f6e2d3644e4a53/recovered.edits/4.seqid to hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/archive/data/default/Group_testCreateMultiRegion/f55a12abc8e0bcbed2f6e2d3644e4a53/recovered.edits/4.seqid 2023-07-24 20:10:49,444 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/b3a80fc52bfa48de445ea8e679dbc14a 2023-07-24 20:10:49,444 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateMultiRegion/f55a12abc8e0bcbed2f6e2d3644e4a53 2023-07-24 20:10:49,444 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived Group_testCreateMultiRegion regions 2023-07-24 20:10:49,448 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=66, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=Group_testCreateMultiRegion 2023-07-24 20:10:49,453 WARN [PEWorker-3] procedure.DeleteTableProcedure(384): Deleting some vestigial 10 rows of Group_testCreateMultiRegion from hbase:meta 2023-07-24 20:10:49,457 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(421): Removing 'Group_testCreateMultiRegion' descriptor. 2023-07-24 20:10:49,460 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=66, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=Group_testCreateMultiRegion 2023-07-24 20:10:49,460 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(411): Removing 'Group_testCreateMultiRegion' from region states. 2023-07-24 20:10:49,460 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCreateMultiRegion,\\x00\\x02\\x04\\x06\\x08,1690229447504.7f557baf61410e30c82ed736835597ae.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690229449460"}]},"ts":"9223372036854775807"} 2023-07-24 20:10:49,460 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCreateMultiRegion,\\x00\"$\u0026(,1690229447504.486c039d584a40408ca57b59a9685da6.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690229449460"}]},"ts":"9223372036854775807"} 2023-07-24 20:10:49,460 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCreateMultiRegion,\\x00BDFH,1690229447504.83406e51fcf8abcdba40d7c315de5777.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690229449460"}]},"ts":"9223372036854775807"} 2023-07-24 20:10:49,460 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCreateMultiRegion,\\x00bdfh,1690229447504.98abe1672c7585d97343038205bd5450.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690229449460"}]},"ts":"9223372036854775807"} 2023-07-24 20:10:49,460 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCreateMultiRegion,\\x00\\x82\\x84\\x86\\x88,1690229447504.8a701529ae84652c9c029ed9bd063968.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690229449460"}]},"ts":"9223372036854775807"} 2023-07-24 20:10:49,461 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCreateMultiRegion,\\x00\\xA2\\xA4\\xA6\\xA8,1690229447504.2df6ec0be254267ea2d62569ac1c609d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690229449460"}]},"ts":"9223372036854775807"} 2023-07-24 20:10:49,461 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCreateMultiRegion,\\x00\\xC2\\xC4\\xC6\\xC8,1690229447504.30fd729e25b9ba973f8749e1094e1710.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690229449460"}]},"ts":"9223372036854775807"} 2023-07-24 20:10:49,461 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCreateMultiRegion,\\x00\\xE2\\xE4\\xE6\\xE8,1690229447504.437177da113d7ca44d1d74b0ae731139.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690229449460"}]},"ts":"9223372036854775807"} 2023-07-24 20:10:49,461 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCreateMultiRegion,\\x01\\x03\\x05\\x07\\x09,1690229447504.b3a80fc52bfa48de445ea8e679dbc14a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690229449460"}]},"ts":"9223372036854775807"} 2023-07-24 20:10:49,461 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCreateMultiRegion,,1690229447504.f55a12abc8e0bcbed2f6e2d3644e4a53.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690229449460"}]},"ts":"9223372036854775807"} 2023-07-24 20:10:49,465 INFO [PEWorker-3] hbase.MetaTableAccessor(1788): Deleted 10 regions from META 2023-07-24 20:10:49,465 DEBUG [PEWorker-3] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 7f557baf61410e30c82ed736835597ae, NAME => 'Group_testCreateMultiRegion,\x00\x02\x04\x06\x08,1690229447504.7f557baf61410e30c82ed736835597ae.', STARTKEY => '\x00\x02\x04\x06\x08', ENDKEY => '\x00"$&('}, {ENCODED => 486c039d584a40408ca57b59a9685da6, NAME => 'Group_testCreateMultiRegion,\x00"$&(,1690229447504.486c039d584a40408ca57b59a9685da6.', STARTKEY => '\x00"$&(', ENDKEY => '\x00BDFH'}, {ENCODED => 83406e51fcf8abcdba40d7c315de5777, NAME => 'Group_testCreateMultiRegion,\x00BDFH,1690229447504.83406e51fcf8abcdba40d7c315de5777.', STARTKEY => '\x00BDFH', ENDKEY => '\x00bdfh'}, {ENCODED => 98abe1672c7585d97343038205bd5450, NAME => 'Group_testCreateMultiRegion,\x00bdfh,1690229447504.98abe1672c7585d97343038205bd5450.', STARTKEY => '\x00bdfh', ENDKEY => '\x00\x82\x84\x86\x88'}, {ENCODED => 8a701529ae84652c9c029ed9bd063968, NAME => 'Group_testCreateMultiRegion,\x00\x82\x84\x86\x88,1690229447504.8a701529ae84652c9c029ed9bd063968.', STARTKEY => '\x00\x82\x84\x86\x88', ENDKEY => '\x00\xA2\xA4\xA6\xA8'}, {ENCODED => 2df6ec0be254267ea2d62569ac1c609d, NAME => 'Group_testCreateMultiRegion,\x00\xA2\xA4\xA6\xA8,1690229447504.2df6ec0be254267ea2d62569ac1c609d.', STARTKEY => '\x00\xA2\xA4\xA6\xA8', ENDKEY => '\x00\xC2\xC4\xC6\xC8'}, {ENCODED => 30fd729e25b9ba973f8749e1094e1710, NAME => 'Group_testCreateMultiRegion,\x00\xC2\xC4\xC6\xC8,1690229447504.30fd729e25b9ba973f8749e1094e1710.', STARTKEY => '\x00\xC2\xC4\xC6\xC8', ENDKEY => '\x00\xE2\xE4\xE6\xE8'}, {ENCODED => 437177da113d7ca44d1d74b0ae731139, NAME => 'Group_testCreateMultiRegion,\x00\xE2\xE4\xE6\xE8,1690229447504.437177da113d7ca44d1d74b0ae731139.', STARTKEY => '\x00\xE2\xE4\xE6\xE8', ENDKEY => '\x01\x03\x05\x07\x09'}, {ENCODED => b3a80fc52bfa48de445ea8e679dbc14a, NAME => 'Group_testCreateMultiRegion,\x01\x03\x05\x07\x09,1690229447504.b3a80fc52bfa48de445ea8e679dbc14a.', STARTKEY => '\x01\x03\x05\x07\x09', ENDKEY => ''}, {ENCODED => f55a12abc8e0bcbed2f6e2d3644e4a53, NAME => 'Group_testCreateMultiRegion,,1690229447504.f55a12abc8e0bcbed2f6e2d3644e4a53.', STARTKEY => '', ENDKEY => '\x00\x02\x04\x06\x08'}] 2023-07-24 20:10:49,465 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(415): Marking 'Group_testCreateMultiRegion' as deleted. 2023-07-24 20:10:49,465 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCreateMultiRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1690229449465"}]},"ts":"9223372036854775807"} 2023-07-24 20:10:49,468 INFO [PEWorker-3] hbase.MetaTableAccessor(1658): Deleted table Group_testCreateMultiRegion state from META 2023-07-24 20:10:49,473 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(130): Finished pid=66, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=Group_testCreateMultiRegion 2023-07-24 20:10:49,476 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=66, state=SUCCESS; DeleteTableProcedure table=Group_testCreateMultiRegion in 101 msec 2023-07-24 20:10:49,485 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=66 2023-07-24 20:10:49,486 INFO [Listener at localhost/44145] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:Group_testCreateMultiRegion, procId: 66 completed 2023-07-24 20:10:49,497 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:49,497 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:49,498 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-24 20:10:49,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-24 20:10:49,499 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-24 20:10:49,503 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-24 20:10:49,503 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-24 20:10:49,504 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-24 20:10:49,509 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:49,510 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-24 20:10:49,512 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-24 20:10:49,517 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase(152): Restoring servers: 0 2023-07-24 20:10:49,519 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-24 20:10:49,522 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:49,523 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:49,524 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-24 20:10:49,526 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-24 20:10:49,530 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:49,530 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:49,534 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:39247] to rsgroup master 2023-07-24 20:10:49,535 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-24 20:10:49,535 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.CallRunner(144): callId: 251 service: MasterService methodName: ExecMasterService size: 119 connection: 172.31.14.131:36610 deadline: 1690230649534, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. 2023-07-24 20:10:49,535 WARN [Listener at localhost/44145] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.afterMethod(TestRSGroupsBasics.java:82) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-24 20:10:49,537 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:10:49,538 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:49,539 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:49,539 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:34483, jenkins-hbase4.apache.org:34917, jenkins-hbase4.apache.org:39791, jenkins-hbase4.apache.org:45209], Tables:[hbase:meta, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-24 20:10:49,540 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-24 20:10:49,540 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-24 20:10:49,560 INFO [Listener at localhost/44145] hbase.ResourceChecker(175): after: rsgroup.TestRSGroupsBasics#testCreateMultiRegion Thread=505 (was 493) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-10 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1540241166_17 at /127.0.0.1:43078 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xb35bd70-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: hconnection-0xb35bd70-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1462415617_17 at /127.0.0.1:45024 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4636fd6e-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=793 (was 753) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=426 (was 394) - SystemLoadAverage LEAK? -, ProcessCount=176 (was 176), AvailableMemoryMB=5836 (was 5926) 2023-07-24 20:10:49,560 WARN [Listener at localhost/44145] hbase.ResourceChecker(130): Thread=505 is superior to 500 2023-07-24 20:10:49,581 INFO [Listener at localhost/44145] hbase.ResourceChecker(147): before: rsgroup.TestRSGroupsBasics#testNamespaceCreateAndAssign Thread=505, OpenFileDescriptor=793, MaxFileDescriptor=60000, SystemLoadAverage=426, ProcessCount=176, AvailableMemoryMB=5834 2023-07-24 20:10:49,581 WARN [Listener at localhost/44145] hbase.ResourceChecker(130): Thread=505 is superior to 500 2023-07-24 20:10:49,581 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase(132): testNamespaceCreateAndAssign 2023-07-24 20:10:49,586 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:49,586 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:49,589 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-24 20:10:49,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-24 20:10:49,589 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-24 20:10:49,590 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-24 20:10:49,590 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-24 20:10:49,591 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-24 20:10:49,595 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:49,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-24 20:10:49,598 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-24 20:10:49,602 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase(152): Restoring servers: 0 2023-07-24 20:10:49,603 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-24 20:10:49,605 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:49,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:49,607 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-24 20:10:49,609 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-24 20:10:49,612 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:49,612 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:49,615 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:39247] to rsgroup master 2023-07-24 20:10:49,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-24 20:10:49,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.CallRunner(144): callId: 279 service: MasterService methodName: ExecMasterService size: 119 connection: 172.31.14.131:36610 deadline: 1690230649614, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. 2023-07-24 20:10:49,615 WARN [Listener at localhost/44145] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.setUpBeforeMethod(TestRSGroupsBase.java:136) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.beforeMethod(TestRSGroupsBasics.java:77) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-24 20:10:49,617 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:10:49,618 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:49,618 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:49,618 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:34483, jenkins-hbase4.apache.org:34917, jenkins-hbase4.apache.org:39791, jenkins-hbase4.apache.org:45209], Tables:[hbase:meta, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-24 20:10:49,619 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-24 20:10:49,619 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-24 20:10:49,619 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBasics(118): testNamespaceCreateAndAssign 2023-07-24 20:10:49,620 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-24 20:10:49,620 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-24 20:10:49,621 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup appInfo 2023-07-24 20:10:49,624 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:49,624 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:49,625 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/appInfo 2023-07-24 20:10:49,631 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 6 2023-07-24 20:10:49,632 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-24 20:10:49,636 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:49,636 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:49,638 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:34483] to rsgroup appInfo 2023-07-24 20:10:49,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:49,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:49,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/appInfo 2023-07-24 20:10:49,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 6 2023-07-24 20:10:49,643 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(286): Moving 0 region(s) to group default, current retry=0 2023-07-24 20:10:49,643 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(261): All regions from [jenkins-hbase4.apache.org,34483,1690229439858] are moved back to default 2023-07-24 20:10:49,643 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(438): Move servers done: default => appInfo 2023-07-24 20:10:49,644 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-24 20:10:49,647 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:49,647 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:49,650 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=appInfo 2023-07-24 20:10:49,650 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-24 20:10:49,656 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.HMaster$15(3014): Client=jenkins//172.31.14.131 creating {NAME => 'Group_foo', hbase.rsgroup.name => 'appInfo'} 2023-07-24 20:10:49,658 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] procedure2.ProcedureExecutor(1029): Stored pid=67, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=Group_foo 2023-07-24 20:10:49,666 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=67 2023-07-24 20:10:49,669 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-07-24 20:10:49,674 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=67, state=SUCCESS; CreateNamespaceProcedure, namespace=Group_foo in 15 msec 2023-07-24 20:10:49,768 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=67 2023-07-24 20:10:49,769 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'Group_foo:Group_testCreateAndAssign', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-24 20:10:49,771 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=Group_foo:Group_testCreateAndAssign 2023-07-24 20:10:49,773 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=68, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=Group_foo:Group_testCreateAndAssign execute state=CREATE_TABLE_PRE_OPERATION 2023-07-24 20:10:49,773 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "Group_foo" qualifier: "Group_testCreateAndAssign" procId is: 68 2023-07-24 20:10:49,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=68 2023-07-24 20:10:49,776 DEBUG [PEWorker-5] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:49,777 DEBUG [PEWorker-5] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:49,777 DEBUG [PEWorker-5] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/appInfo 2023-07-24 20:10:49,778 DEBUG [PEWorker-5] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 6 2023-07-24 20:10:49,780 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=68, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=Group_foo:Group_testCreateAndAssign execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-24 20:10:49,782 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/Group_foo/Group_testCreateAndAssign/bcf9416b673645032149b39146293566 2023-07-24 20:10:49,783 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/Group_foo/Group_testCreateAndAssign/bcf9416b673645032149b39146293566 empty. 2023-07-24 20:10:49,784 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/Group_foo/Group_testCreateAndAssign/bcf9416b673645032149b39146293566 2023-07-24 20:10:49,784 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived Group_foo:Group_testCreateAndAssign regions 2023-07-24 20:10:49,804 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/Group_foo/Group_testCreateAndAssign/.tabledesc/.tableinfo.0000000001 2023-07-24 20:10:49,806 INFO [RegionOpenAndInit-Group_foo:Group_testCreateAndAssign-pool-0] regionserver.HRegion(7675): creating {ENCODED => bcf9416b673645032149b39146293566, NAME => 'Group_foo:Group_testCreateAndAssign,,1690229449769.bcf9416b673645032149b39146293566.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='Group_foo:Group_testCreateAndAssign', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp 2023-07-24 20:10:49,839 DEBUG [RegionOpenAndInit-Group_foo:Group_testCreateAndAssign-pool-0] regionserver.HRegion(866): Instantiated Group_foo:Group_testCreateAndAssign,,1690229449769.bcf9416b673645032149b39146293566.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:49,839 DEBUG [RegionOpenAndInit-Group_foo:Group_testCreateAndAssign-pool-0] regionserver.HRegion(1604): Closing bcf9416b673645032149b39146293566, disabling compactions & flushes 2023-07-24 20:10:49,840 INFO [RegionOpenAndInit-Group_foo:Group_testCreateAndAssign-pool-0] regionserver.HRegion(1626): Closing region Group_foo:Group_testCreateAndAssign,,1690229449769.bcf9416b673645032149b39146293566. 2023-07-24 20:10:49,840 DEBUG [RegionOpenAndInit-Group_foo:Group_testCreateAndAssign-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_foo:Group_testCreateAndAssign,,1690229449769.bcf9416b673645032149b39146293566. 2023-07-24 20:10:49,840 DEBUG [RegionOpenAndInit-Group_foo:Group_testCreateAndAssign-pool-0] regionserver.HRegion(1714): Acquired close lock on Group_foo:Group_testCreateAndAssign,,1690229449769.bcf9416b673645032149b39146293566. after waiting 0 ms 2023-07-24 20:10:49,840 DEBUG [RegionOpenAndInit-Group_foo:Group_testCreateAndAssign-pool-0] regionserver.HRegion(1724): Updates disabled for region Group_foo:Group_testCreateAndAssign,,1690229449769.bcf9416b673645032149b39146293566. 2023-07-24 20:10:49,840 INFO [RegionOpenAndInit-Group_foo:Group_testCreateAndAssign-pool-0] regionserver.HRegion(1838): Closed Group_foo:Group_testCreateAndAssign,,1690229449769.bcf9416b673645032149b39146293566. 2023-07-24 20:10:49,840 DEBUG [RegionOpenAndInit-Group_foo:Group_testCreateAndAssign-pool-0] regionserver.HRegion(1558): Region close journal for bcf9416b673645032149b39146293566: 2023-07-24 20:10:49,844 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=68, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=Group_foo:Group_testCreateAndAssign execute state=CREATE_TABLE_ADD_TO_META 2023-07-24 20:10:49,846 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"Group_foo:Group_testCreateAndAssign,,1690229449769.bcf9416b673645032149b39146293566.","families":{"info":[{"qualifier":"regioninfo","vlen":61,"tag":[],"timestamp":"1690229449846"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229449846"}]},"ts":"1690229449846"} 2023-07-24 20:10:49,848 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-24 20:10:49,850 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=68, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=Group_foo:Group_testCreateAndAssign execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-24 20:10:49,850 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_foo:Group_testCreateAndAssign","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690229449850"}]},"ts":"1690229449850"} 2023-07-24 20:10:49,852 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=Group_foo:Group_testCreateAndAssign, state=ENABLING in hbase:meta 2023-07-24 20:10:49,855 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=69, ppid=68, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_foo:Group_testCreateAndAssign, region=bcf9416b673645032149b39146293566, ASSIGN}] 2023-07-24 20:10:49,858 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=69, ppid=68, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_foo:Group_testCreateAndAssign, region=bcf9416b673645032149b39146293566, ASSIGN 2023-07-24 20:10:49,860 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=69, ppid=68, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=Group_foo:Group_testCreateAndAssign, region=bcf9416b673645032149b39146293566, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34483,1690229439858; forceNewPlan=false, retain=false 2023-07-24 20:10:49,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=68 2023-07-24 20:10:50,012 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=69 updating hbase:meta row=bcf9416b673645032149b39146293566, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:50,012 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_foo:Group_testCreateAndAssign,,1690229449769.bcf9416b673645032149b39146293566.","families":{"info":[{"qualifier":"regioninfo","vlen":61,"tag":[],"timestamp":"1690229450012"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229450012"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229450012"}]},"ts":"1690229450012"} 2023-07-24 20:10:50,015 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE; OpenRegionProcedure bcf9416b673645032149b39146293566, server=jenkins-hbase4.apache.org,34483,1690229439858}] 2023-07-24 20:10:50,088 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=68 2023-07-24 20:10:50,174 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open Group_foo:Group_testCreateAndAssign,,1690229449769.bcf9416b673645032149b39146293566. 2023-07-24 20:10:50,174 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => bcf9416b673645032149b39146293566, NAME => 'Group_foo:Group_testCreateAndAssign,,1690229449769.bcf9416b673645032149b39146293566.', STARTKEY => '', ENDKEY => ''} 2023-07-24 20:10:50,174 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table Group_testCreateAndAssign bcf9416b673645032149b39146293566 2023-07-24 20:10:50,174 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated Group_foo:Group_testCreateAndAssign,,1690229449769.bcf9416b673645032149b39146293566.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:50,175 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for bcf9416b673645032149b39146293566 2023-07-24 20:10:50,175 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for bcf9416b673645032149b39146293566 2023-07-24 20:10:50,176 INFO [StoreOpener-bcf9416b673645032149b39146293566-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f of region bcf9416b673645032149b39146293566 2023-07-24 20:10:50,181 DEBUG [StoreOpener-bcf9416b673645032149b39146293566-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/Group_foo/Group_testCreateAndAssign/bcf9416b673645032149b39146293566/f 2023-07-24 20:10:50,181 DEBUG [StoreOpener-bcf9416b673645032149b39146293566-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/Group_foo/Group_testCreateAndAssign/bcf9416b673645032149b39146293566/f 2023-07-24 20:10:50,181 INFO [StoreOpener-bcf9416b673645032149b39146293566-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bcf9416b673645032149b39146293566 columnFamilyName f 2023-07-24 20:10:50,182 INFO [StoreOpener-bcf9416b673645032149b39146293566-1] regionserver.HStore(310): Store=bcf9416b673645032149b39146293566/f, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:10:50,184 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/Group_foo/Group_testCreateAndAssign/bcf9416b673645032149b39146293566 2023-07-24 20:10:50,184 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/Group_foo/Group_testCreateAndAssign/bcf9416b673645032149b39146293566 2023-07-24 20:10:50,195 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for bcf9416b673645032149b39146293566 2023-07-24 20:10:50,204 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/Group_foo/Group_testCreateAndAssign/bcf9416b673645032149b39146293566/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 20:10:50,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened bcf9416b673645032149b39146293566; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=10228751680, jitterRate=-0.04737326502799988}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 20:10:50,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for bcf9416b673645032149b39146293566: 2023-07-24 20:10:50,207 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for Group_foo:Group_testCreateAndAssign,,1690229449769.bcf9416b673645032149b39146293566., pid=70, masterSystemTime=1690229450169 2023-07-24 20:10:50,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for Group_foo:Group_testCreateAndAssign,,1690229449769.bcf9416b673645032149b39146293566. 2023-07-24 20:10:50,210 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened Group_foo:Group_testCreateAndAssign,,1690229449769.bcf9416b673645032149b39146293566. 2023-07-24 20:10:50,210 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=69 updating hbase:meta row=bcf9416b673645032149b39146293566, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:50,211 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"Group_foo:Group_testCreateAndAssign,,1690229449769.bcf9416b673645032149b39146293566.","families":{"info":[{"qualifier":"regioninfo","vlen":61,"tag":[],"timestamp":"1690229450210"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690229450210"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690229450210"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690229450210"}]},"ts":"1690229450210"} 2023-07-24 20:10:50,221 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-07-24 20:10:50,221 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; OpenRegionProcedure bcf9416b673645032149b39146293566, server=jenkins-hbase4.apache.org,34483,1690229439858 in 198 msec 2023-07-24 20:10:50,224 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=69, resume processing ppid=68 2023-07-24 20:10:50,224 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=69, ppid=68, state=SUCCESS; TransitRegionStateProcedure table=Group_foo:Group_testCreateAndAssign, region=bcf9416b673645032149b39146293566, ASSIGN in 366 msec 2023-07-24 20:10:50,225 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=68, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=Group_foo:Group_testCreateAndAssign execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-24 20:10:50,225 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_foo:Group_testCreateAndAssign","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690229450225"}]},"ts":"1690229450225"} 2023-07-24 20:10:50,228 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=Group_foo:Group_testCreateAndAssign, state=ENABLED in hbase:meta 2023-07-24 20:10:50,233 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=68, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=Group_foo:Group_testCreateAndAssign execute state=CREATE_TABLE_POST_OPERATION 2023-07-24 20:10:50,237 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; CreateTableProcedure table=Group_foo:Group_testCreateAndAssign in 464 msec 2023-07-24 20:10:50,390 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=68 2023-07-24 20:10:50,390 INFO [Listener at localhost/44145] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: Group_foo:Group_testCreateAndAssign, procId: 68 completed 2023-07-24 20:10:50,391 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:10:50,396 INFO [Listener at localhost/44145] client.HBaseAdmin$15(890): Started disable of Group_foo:Group_testCreateAndAssign 2023-07-24 20:10:50,396 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable Group_foo:Group_testCreateAndAssign 2023-07-24 20:10:50,397 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] procedure2.ProcedureExecutor(1029): Stored pid=71, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=Group_foo:Group_testCreateAndAssign 2023-07-24 20:10:50,400 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=71 2023-07-24 20:10:50,401 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_foo:Group_testCreateAndAssign","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690229450400"}]},"ts":"1690229450400"} 2023-07-24 20:10:50,402 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=Group_foo:Group_testCreateAndAssign, state=DISABLING in hbase:meta 2023-07-24 20:10:50,404 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set Group_foo:Group_testCreateAndAssign to state=DISABLING 2023-07-24 20:10:50,405 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_foo:Group_testCreateAndAssign, region=bcf9416b673645032149b39146293566, UNASSIGN}] 2023-07-24 20:10:50,406 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_foo:Group_testCreateAndAssign, region=bcf9416b673645032149b39146293566, UNASSIGN 2023-07-24 20:10:50,407 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=bcf9416b673645032149b39146293566, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:50,407 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_foo:Group_testCreateAndAssign,,1690229449769.bcf9416b673645032149b39146293566.","families":{"info":[{"qualifier":"regioninfo","vlen":61,"tag":[],"timestamp":"1690229450407"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229450407"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229450407"}]},"ts":"1690229450407"} 2023-07-24 20:10:50,409 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE; CloseRegionProcedure bcf9416b673645032149b39146293566, server=jenkins-hbase4.apache.org,34483,1690229439858}] 2023-07-24 20:10:50,501 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=71 2023-07-24 20:10:50,561 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close bcf9416b673645032149b39146293566 2023-07-24 20:10:50,562 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing bcf9416b673645032149b39146293566, disabling compactions & flushes 2023-07-24 20:10:50,562 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region Group_foo:Group_testCreateAndAssign,,1690229449769.bcf9416b673645032149b39146293566. 2023-07-24 20:10:50,562 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_foo:Group_testCreateAndAssign,,1690229449769.bcf9416b673645032149b39146293566. 2023-07-24 20:10:50,562 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on Group_foo:Group_testCreateAndAssign,,1690229449769.bcf9416b673645032149b39146293566. after waiting 0 ms 2023-07-24 20:10:50,562 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region Group_foo:Group_testCreateAndAssign,,1690229449769.bcf9416b673645032149b39146293566. 2023-07-24 20:10:50,567 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/Group_foo/Group_testCreateAndAssign/bcf9416b673645032149b39146293566/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-24 20:10:50,568 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed Group_foo:Group_testCreateAndAssign,,1690229449769.bcf9416b673645032149b39146293566. 2023-07-24 20:10:50,568 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for bcf9416b673645032149b39146293566: 2023-07-24 20:10:50,570 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed bcf9416b673645032149b39146293566 2023-07-24 20:10:50,570 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=bcf9416b673645032149b39146293566, regionState=CLOSED 2023-07-24 20:10:50,570 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"Group_foo:Group_testCreateAndAssign,,1690229449769.bcf9416b673645032149b39146293566.","families":{"info":[{"qualifier":"regioninfo","vlen":61,"tag":[],"timestamp":"1690229450570"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229450570"}]},"ts":"1690229450570"} 2023-07-24 20:10:50,574 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-07-24 20:10:50,574 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; CloseRegionProcedure bcf9416b673645032149b39146293566, server=jenkins-hbase4.apache.org,34483,1690229439858 in 163 msec 2023-07-24 20:10:50,579 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=72, resume processing ppid=71 2023-07-24 20:10:50,579 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=72, ppid=71, state=SUCCESS; TransitRegionStateProcedure table=Group_foo:Group_testCreateAndAssign, region=bcf9416b673645032149b39146293566, UNASSIGN in 169 msec 2023-07-24 20:10:50,580 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_foo:Group_testCreateAndAssign","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690229450580"}]},"ts":"1690229450580"} 2023-07-24 20:10:50,582 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=Group_foo:Group_testCreateAndAssign, state=DISABLED in hbase:meta 2023-07-24 20:10:50,583 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set Group_foo:Group_testCreateAndAssign to state=DISABLED 2023-07-24 20:10:50,586 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, state=SUCCESS; DisableTableProcedure table=Group_foo:Group_testCreateAndAssign in 188 msec 2023-07-24 20:10:50,703 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=71 2023-07-24 20:10:50,703 INFO [Listener at localhost/44145] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: Group_foo:Group_testCreateAndAssign, procId: 71 completed 2023-07-24 20:10:50,704 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete Group_foo:Group_testCreateAndAssign 2023-07-24 20:10:50,705 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] procedure2.ProcedureExecutor(1029): Stored pid=74, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=Group_foo:Group_testCreateAndAssign 2023-07-24 20:10:50,707 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=74, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=Group_foo:Group_testCreateAndAssign 2023-07-24 20:10:50,707 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint(577): Removing deleted table 'Group_foo:Group_testCreateAndAssign' from rsgroup 'appInfo' 2023-07-24 20:10:50,708 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=74, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=Group_foo:Group_testCreateAndAssign 2023-07-24 20:10:50,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:50,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:50,711 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/appInfo 2023-07-24 20:10:50,711 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 6 2023-07-24 20:10:50,713 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/Group_foo/Group_testCreateAndAssign/bcf9416b673645032149b39146293566 2023-07-24 20:10:50,714 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=74 2023-07-24 20:10:50,715 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/Group_foo/Group_testCreateAndAssign/bcf9416b673645032149b39146293566/f, FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/Group_foo/Group_testCreateAndAssign/bcf9416b673645032149b39146293566/recovered.edits] 2023-07-24 20:10:50,720 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/Group_foo/Group_testCreateAndAssign/bcf9416b673645032149b39146293566/recovered.edits/4.seqid to hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/archive/data/Group_foo/Group_testCreateAndAssign/bcf9416b673645032149b39146293566/recovered.edits/4.seqid 2023-07-24 20:10:50,721 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/Group_foo/Group_testCreateAndAssign/bcf9416b673645032149b39146293566 2023-07-24 20:10:50,721 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived Group_foo:Group_testCreateAndAssign regions 2023-07-24 20:10:50,724 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=74, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=Group_foo:Group_testCreateAndAssign 2023-07-24 20:10:50,726 WARN [PEWorker-3] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of Group_foo:Group_testCreateAndAssign from hbase:meta 2023-07-24 20:10:50,728 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(421): Removing 'Group_foo:Group_testCreateAndAssign' descriptor. 2023-07-24 20:10:50,730 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=74, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=Group_foo:Group_testCreateAndAssign 2023-07-24 20:10:50,730 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(411): Removing 'Group_foo:Group_testCreateAndAssign' from region states. 2023-07-24 20:10:50,730 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_foo:Group_testCreateAndAssign,,1690229449769.bcf9416b673645032149b39146293566.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690229450730"}]},"ts":"9223372036854775807"} 2023-07-24 20:10:50,732 INFO [PEWorker-3] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-07-24 20:10:50,732 DEBUG [PEWorker-3] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => bcf9416b673645032149b39146293566, NAME => 'Group_foo:Group_testCreateAndAssign,,1690229449769.bcf9416b673645032149b39146293566.', STARTKEY => '', ENDKEY => ''}] 2023-07-24 20:10:50,732 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(415): Marking 'Group_foo:Group_testCreateAndAssign' as deleted. 2023-07-24 20:10:50,732 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_foo:Group_testCreateAndAssign","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1690229450732"}]},"ts":"9223372036854775807"} 2023-07-24 20:10:50,734 INFO [PEWorker-3] hbase.MetaTableAccessor(1658): Deleted table Group_foo:Group_testCreateAndAssign state from META 2023-07-24 20:10:50,737 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(130): Finished pid=74, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=Group_foo:Group_testCreateAndAssign 2023-07-24 20:10:50,739 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=74, state=SUCCESS; DeleteTableProcedure table=Group_foo:Group_testCreateAndAssign in 33 msec 2023-07-24 20:10:50,815 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=74 2023-07-24 20:10:50,816 INFO [Listener at localhost/44145] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: Group_foo:Group_testCreateAndAssign, procId: 74 completed 2023-07-24 20:10:50,824 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.HMaster$17(3086): Client=jenkins//172.31.14.131 delete Group_foo 2023-07-24 20:10:50,834 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] procedure2.ProcedureExecutor(1029): Stored pid=75, state=RUNNABLE:DELETE_NAMESPACE_PREPARE; DeleteNamespaceProcedure, namespace=Group_foo 2023-07-24 20:10:50,836 INFO [PEWorker-2] procedure.DeleteNamespaceProcedure(73): pid=75, state=RUNNABLE:DELETE_NAMESPACE_PREPARE, locked=true; DeleteNamespaceProcedure, namespace=Group_foo 2023-07-24 20:10:50,840 INFO [PEWorker-2] procedure.DeleteNamespaceProcedure(73): pid=75, state=RUNNABLE:DELETE_NAMESPACE_DELETE_FROM_NS_TABLE, locked=true; DeleteNamespaceProcedure, namespace=Group_foo 2023-07-24 20:10:50,843 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=75 2023-07-24 20:10:50,843 INFO [PEWorker-2] procedure.DeleteNamespaceProcedure(73): pid=75, state=RUNNABLE:DELETE_NAMESPACE_REMOVE_FROM_ZK, locked=true; DeleteNamespaceProcedure, namespace=Group_foo 2023-07-24 20:10:50,845 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/namespace/Group_foo 2023-07-24 20:10:50,845 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-07-24 20:10:50,846 INFO [PEWorker-2] procedure.DeleteNamespaceProcedure(73): pid=75, state=RUNNABLE:DELETE_NAMESPACE_DELETE_DIRECTORIES, locked=true; DeleteNamespaceProcedure, namespace=Group_foo 2023-07-24 20:10:50,848 INFO [PEWorker-2] procedure.DeleteNamespaceProcedure(73): pid=75, state=RUNNABLE:DELETE_NAMESPACE_REMOVE_NAMESPACE_QUOTA, locked=true; DeleteNamespaceProcedure, namespace=Group_foo 2023-07-24 20:10:50,849 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=75, state=SUCCESS; DeleteNamespaceProcedure, namespace=Group_foo in 24 msec 2023-07-24 20:10:50,944 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=75 2023-07-24 20:10:50,946 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:50,946 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:50,947 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-24 20:10:50,947 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-24 20:10:50,947 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-24 20:10:50,948 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-24 20:10:50,948 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-24 20:10:50,950 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-24 20:10:50,954 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:50,954 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/appInfo 2023-07-24 20:10:50,955 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 5 2023-07-24 20:10:50,960 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-24 20:10:50,961 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-24 20:10:50,961 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-24 20:10:50,961 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-24 20:10:50,962 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:34483] to rsgroup default 2023-07-24 20:10:50,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:50,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/appInfo 2023-07-24 20:10:50,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-24 20:10:50,966 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(286): Moving 0 region(s) to group appInfo, current retry=0 2023-07-24 20:10:50,966 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(261): All regions from [jenkins-hbase4.apache.org,34483,1690229439858] are moved back to appInfo 2023-07-24 20:10:50,966 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(438): Move servers done: appInfo => default 2023-07-24 20:10:50,966 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-24 20:10:50,967 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup appInfo 2023-07-24 20:10:50,970 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:50,971 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-24 20:10:50,972 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-24 20:10:50,975 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase(152): Restoring servers: 0 2023-07-24 20:10:50,975 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-24 20:10:50,977 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:50,978 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:50,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-24 20:10:50,985 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-24 20:10:50,987 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:50,987 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:50,990 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:39247] to rsgroup master 2023-07-24 20:10:50,990 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-24 20:10:50,990 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.CallRunner(144): callId: 368 service: MasterService methodName: ExecMasterService size: 119 connection: 172.31.14.131:36610 deadline: 1690230650990, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. 2023-07-24 20:10:50,991 WARN [Listener at localhost/44145] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.afterMethod(TestRSGroupsBasics.java:82) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-24 20:10:50,992 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:10:50,993 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:50,993 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:50,994 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:34483, jenkins-hbase4.apache.org:34917, jenkins-hbase4.apache.org:39791, jenkins-hbase4.apache.org:45209], Tables:[hbase:meta, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-24 20:10:50,994 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-24 20:10:50,994 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-24 20:10:51,011 INFO [Listener at localhost/44145] hbase.ResourceChecker(175): after: rsgroup.TestRSGroupsBasics#testNamespaceCreateAndAssign Thread=505 (was 505), OpenFileDescriptor=792 (was 793), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=426 (was 426), ProcessCount=176 (was 176), AvailableMemoryMB=5831 (was 5834) 2023-07-24 20:10:51,011 WARN [Listener at localhost/44145] hbase.ResourceChecker(130): Thread=505 is superior to 500 2023-07-24 20:10:51,029 INFO [Listener at localhost/44145] hbase.ResourceChecker(147): before: rsgroup.TestRSGroupsBasics#testCreateAndDrop Thread=505, OpenFileDescriptor=792, MaxFileDescriptor=60000, SystemLoadAverage=426, ProcessCount=176, AvailableMemoryMB=5830 2023-07-24 20:10:51,029 WARN [Listener at localhost/44145] hbase.ResourceChecker(130): Thread=505 is superior to 500 2023-07-24 20:10:51,029 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase(132): testCreateAndDrop 2023-07-24 20:10:51,034 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:51,034 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:51,035 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-24 20:10:51,036 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-24 20:10:51,036 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-24 20:10:51,037 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-24 20:10:51,037 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-24 20:10:51,038 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-24 20:10:51,042 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:51,043 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-24 20:10:51,044 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-24 20:10:51,047 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase(152): Restoring servers: 0 2023-07-24 20:10:51,048 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-24 20:10:51,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:51,051 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:51,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-24 20:10:51,055 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-24 20:10:51,059 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:51,059 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:51,062 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:39247] to rsgroup master 2023-07-24 20:10:51,062 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-24 20:10:51,062 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.CallRunner(144): callId: 396 service: MasterService methodName: ExecMasterService size: 119 connection: 172.31.14.131:36610 deadline: 1690230651061, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. 2023-07-24 20:10:51,062 WARN [Listener at localhost/44145] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.setUpBeforeMethod(TestRSGroupsBase.java:136) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.beforeMethod(TestRSGroupsBasics.java:77) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-24 20:10:51,064 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:10:51,065 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:51,065 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:51,071 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:34483, jenkins-hbase4.apache.org:34917, jenkins-hbase4.apache.org:39791, jenkins-hbase4.apache.org:45209], Tables:[hbase:meta, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-24 20:10:51,072 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-24 20:10:51,073 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-24 20:10:51,075 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'Group_testCreateAndDrop', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '1'}}, {NAME => 'cf', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-24 20:10:51,076 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] procedure2.ProcedureExecutor(1029): Stored pid=76, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=Group_testCreateAndDrop 2023-07-24 20:10:51,079 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=76, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=Group_testCreateAndDrop execute state=CREATE_TABLE_PRE_OPERATION 2023-07-24 20:10:51,080 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "Group_testCreateAndDrop" procId is: 76 2023-07-24 20:10:51,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=76 2023-07-24 20:10:51,082 DEBUG [PEWorker-4] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:51,082 DEBUG [PEWorker-4] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:51,083 DEBUG [PEWorker-4] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-24 20:10:51,085 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=76, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=Group_testCreateAndDrop execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-24 20:10:51,088 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateAndDrop/2febae84cf14582e40855cc44673eaf2 2023-07-24 20:10:51,088 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateAndDrop/2febae84cf14582e40855cc44673eaf2 empty. 2023-07-24 20:10:51,089 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateAndDrop/2febae84cf14582e40855cc44673eaf2 2023-07-24 20:10:51,089 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived Group_testCreateAndDrop regions 2023-07-24 20:10:51,142 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateAndDrop/.tabledesc/.tableinfo.0000000001 2023-07-24 20:10:51,151 INFO [RegionOpenAndInit-Group_testCreateAndDrop-pool-0] regionserver.HRegion(7675): creating {ENCODED => 2febae84cf14582e40855cc44673eaf2, NAME => 'Group_testCreateAndDrop,,1690229451075.2febae84cf14582e40855cc44673eaf2.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='Group_testCreateAndDrop', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '1'}}, {NAME => 'cf', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp 2023-07-24 20:10:51,168 DEBUG [RegionOpenAndInit-Group_testCreateAndDrop-pool-0] regionserver.HRegion(866): Instantiated Group_testCreateAndDrop,,1690229451075.2febae84cf14582e40855cc44673eaf2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:51,168 DEBUG [RegionOpenAndInit-Group_testCreateAndDrop-pool-0] regionserver.HRegion(1604): Closing 2febae84cf14582e40855cc44673eaf2, disabling compactions & flushes 2023-07-24 20:10:51,168 INFO [RegionOpenAndInit-Group_testCreateAndDrop-pool-0] regionserver.HRegion(1626): Closing region Group_testCreateAndDrop,,1690229451075.2febae84cf14582e40855cc44673eaf2. 2023-07-24 20:10:51,168 DEBUG [RegionOpenAndInit-Group_testCreateAndDrop-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateAndDrop,,1690229451075.2febae84cf14582e40855cc44673eaf2. 2023-07-24 20:10:51,168 DEBUG [RegionOpenAndInit-Group_testCreateAndDrop-pool-0] regionserver.HRegion(1714): Acquired close lock on Group_testCreateAndDrop,,1690229451075.2febae84cf14582e40855cc44673eaf2. after waiting 0 ms 2023-07-24 20:10:51,169 DEBUG [RegionOpenAndInit-Group_testCreateAndDrop-pool-0] regionserver.HRegion(1724): Updates disabled for region Group_testCreateAndDrop,,1690229451075.2febae84cf14582e40855cc44673eaf2. 2023-07-24 20:10:51,169 INFO [RegionOpenAndInit-Group_testCreateAndDrop-pool-0] regionserver.HRegion(1838): Closed Group_testCreateAndDrop,,1690229451075.2febae84cf14582e40855cc44673eaf2. 2023-07-24 20:10:51,169 DEBUG [RegionOpenAndInit-Group_testCreateAndDrop-pool-0] regionserver.HRegion(1558): Region close journal for 2febae84cf14582e40855cc44673eaf2: 2023-07-24 20:10:51,171 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=76, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=Group_testCreateAndDrop execute state=CREATE_TABLE_ADD_TO_META 2023-07-24 20:10:51,172 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"Group_testCreateAndDrop,,1690229451075.2febae84cf14582e40855cc44673eaf2.","families":{"info":[{"qualifier":"regioninfo","vlen":57,"tag":[],"timestamp":"1690229451172"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229451172"}]},"ts":"1690229451172"} 2023-07-24 20:10:51,173 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-24 20:10:51,174 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=76, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=Group_testCreateAndDrop execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-24 20:10:51,174 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCreateAndDrop","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690229451174"}]},"ts":"1690229451174"} 2023-07-24 20:10:51,176 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCreateAndDrop, state=ENABLING in hbase:meta 2023-07-24 20:10:51,182 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-24 20:10:51,182 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 20:10:51,182 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 20:10:51,182 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 20:10:51,183 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-24 20:10:51,183 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 20:10:51,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=76 2023-07-24 20:10:51,183 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateAndDrop, region=2febae84cf14582e40855cc44673eaf2, ASSIGN}] 2023-07-24 20:10:51,185 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=77, ppid=76, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCreateAndDrop, region=2febae84cf14582e40855cc44673eaf2, ASSIGN 2023-07-24 20:10:51,185 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=77, ppid=76, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=Group_testCreateAndDrop, region=2febae84cf14582e40855cc44673eaf2, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45209,1690229445078; forceNewPlan=false, retain=false 2023-07-24 20:10:51,336 INFO [jenkins-hbase4:39247] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-24 20:10:51,337 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=77 updating hbase:meta row=2febae84cf14582e40855cc44673eaf2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:51,337 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateAndDrop,,1690229451075.2febae84cf14582e40855cc44673eaf2.","families":{"info":[{"qualifier":"regioninfo","vlen":57,"tag":[],"timestamp":"1690229451337"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229451337"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229451337"}]},"ts":"1690229451337"} 2023-07-24 20:10:51,339 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=77, state=RUNNABLE; OpenRegionProcedure 2febae84cf14582e40855cc44673eaf2, server=jenkins-hbase4.apache.org,45209,1690229445078}] 2023-07-24 20:10:51,384 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=76 2023-07-24 20:10:51,631 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open Group_testCreateAndDrop,,1690229451075.2febae84cf14582e40855cc44673eaf2. 2023-07-24 20:10:51,631 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2febae84cf14582e40855cc44673eaf2, NAME => 'Group_testCreateAndDrop,,1690229451075.2febae84cf14582e40855cc44673eaf2.', STARTKEY => '', ENDKEY => ''} 2023-07-24 20:10:51,631 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table Group_testCreateAndDrop 2febae84cf14582e40855cc44673eaf2 2023-07-24 20:10:51,632 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated Group_testCreateAndDrop,,1690229451075.2febae84cf14582e40855cc44673eaf2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:51,632 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 2febae84cf14582e40855cc44673eaf2 2023-07-24 20:10:51,632 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 2febae84cf14582e40855cc44673eaf2 2023-07-24 20:10:51,633 INFO [StoreOpener-2febae84cf14582e40855cc44673eaf2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf of region 2febae84cf14582e40855cc44673eaf2 2023-07-24 20:10:51,636 DEBUG [StoreOpener-2febae84cf14582e40855cc44673eaf2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateAndDrop/2febae84cf14582e40855cc44673eaf2/cf 2023-07-24 20:10:51,636 DEBUG [StoreOpener-2febae84cf14582e40855cc44673eaf2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateAndDrop/2febae84cf14582e40855cc44673eaf2/cf 2023-07-24 20:10:51,637 INFO [StoreOpener-2febae84cf14582e40855cc44673eaf2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2febae84cf14582e40855cc44673eaf2 columnFamilyName cf 2023-07-24 20:10:51,638 INFO [StoreOpener-2febae84cf14582e40855cc44673eaf2-1] regionserver.HStore(310): Store=2febae84cf14582e40855cc44673eaf2/cf, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:10:51,639 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateAndDrop/2febae84cf14582e40855cc44673eaf2 2023-07-24 20:10:51,639 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateAndDrop/2febae84cf14582e40855cc44673eaf2 2023-07-24 20:10:51,642 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 2febae84cf14582e40855cc44673eaf2 2023-07-24 20:10:51,645 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateAndDrop/2febae84cf14582e40855cc44673eaf2/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 20:10:51,646 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 2febae84cf14582e40855cc44673eaf2; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=11742953120, jitterRate=0.09364773333072662}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 20:10:51,646 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 2febae84cf14582e40855cc44673eaf2: 2023-07-24 20:10:51,647 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for Group_testCreateAndDrop,,1690229451075.2febae84cf14582e40855cc44673eaf2., pid=78, masterSystemTime=1690229451622 2023-07-24 20:10:51,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for Group_testCreateAndDrop,,1690229451075.2febae84cf14582e40855cc44673eaf2. 2023-07-24 20:10:51,650 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened Group_testCreateAndDrop,,1690229451075.2febae84cf14582e40855cc44673eaf2. 2023-07-24 20:10:51,651 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=77 updating hbase:meta row=2febae84cf14582e40855cc44673eaf2, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:51,651 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"Group_testCreateAndDrop,,1690229451075.2febae84cf14582e40855cc44673eaf2.","families":{"info":[{"qualifier":"regioninfo","vlen":57,"tag":[],"timestamp":"1690229451651"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690229451651"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690229451651"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690229451651"}]},"ts":"1690229451651"} 2023-07-24 20:10:51,656 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=77 2023-07-24 20:10:51,656 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=77, state=SUCCESS; OpenRegionProcedure 2febae84cf14582e40855cc44673eaf2, server=jenkins-hbase4.apache.org,45209,1690229445078 in 314 msec 2023-07-24 20:10:51,658 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-07-24 20:10:51,658 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateAndDrop, region=2febae84cf14582e40855cc44673eaf2, ASSIGN in 473 msec 2023-07-24 20:10:51,659 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=76, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=Group_testCreateAndDrop execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-24 20:10:51,659 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCreateAndDrop","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690229451659"}]},"ts":"1690229451659"} 2023-07-24 20:10:51,660 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCreateAndDrop, state=ENABLED in hbase:meta 2023-07-24 20:10:51,667 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=76, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=Group_testCreateAndDrop execute state=CREATE_TABLE_POST_OPERATION 2023-07-24 20:10:51,669 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=76, state=SUCCESS; CreateTableProcedure table=Group_testCreateAndDrop in 592 msec 2023-07-24 20:10:51,685 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=76 2023-07-24 20:10:51,686 INFO [Listener at localhost/44145] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:Group_testCreateAndDrop, procId: 76 completed 2023-07-24 20:10:51,686 DEBUG [Listener at localhost/44145] hbase.HBaseTestingUtility(3430): Waiting until all regions of table Group_testCreateAndDrop get assigned. Timeout = 60000ms 2023-07-24 20:10:51,686 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:10:51,692 INFO [Listener at localhost/44145] hbase.HBaseTestingUtility(3484): All regions for table Group_testCreateAndDrop assigned to meta. Checking AM states. 2023-07-24 20:10:51,692 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:10:51,693 INFO [Listener at localhost/44145] hbase.HBaseTestingUtility(3504): All regions for table Group_testCreateAndDrop assigned. 2023-07-24 20:10:51,693 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:10:51,696 INFO [Listener at localhost/44145] client.HBaseAdmin$15(890): Started disable of Group_testCreateAndDrop 2023-07-24 20:10:51,697 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable Group_testCreateAndDrop 2023-07-24 20:10:51,698 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=Group_testCreateAndDrop 2023-07-24 20:10:51,700 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-07-24 20:10:51,701 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCreateAndDrop","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690229451701"}]},"ts":"1690229451701"} 2023-07-24 20:10:51,702 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCreateAndDrop, state=DISABLING in hbase:meta 2023-07-24 20:10:51,704 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set Group_testCreateAndDrop to state=DISABLING 2023-07-24 20:10:51,705 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateAndDrop, region=2febae84cf14582e40855cc44673eaf2, UNASSIGN}] 2023-07-24 20:10:51,707 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCreateAndDrop, region=2febae84cf14582e40855cc44673eaf2, UNASSIGN 2023-07-24 20:10:51,708 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=2febae84cf14582e40855cc44673eaf2, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:51,708 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCreateAndDrop,,1690229451075.2febae84cf14582e40855cc44673eaf2.","families":{"info":[{"qualifier":"regioninfo","vlen":57,"tag":[],"timestamp":"1690229451708"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229451708"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229451708"}]},"ts":"1690229451708"} 2023-07-24 20:10:51,710 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=81, ppid=80, state=RUNNABLE; CloseRegionProcedure 2febae84cf14582e40855cc44673eaf2, server=jenkins-hbase4.apache.org,45209,1690229445078}] 2023-07-24 20:10:51,802 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-07-24 20:10:51,862 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 2febae84cf14582e40855cc44673eaf2 2023-07-24 20:10:51,863 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 2febae84cf14582e40855cc44673eaf2, disabling compactions & flushes 2023-07-24 20:10:51,863 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region Group_testCreateAndDrop,,1690229451075.2febae84cf14582e40855cc44673eaf2. 2023-07-24 20:10:51,863 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCreateAndDrop,,1690229451075.2febae84cf14582e40855cc44673eaf2. 2023-07-24 20:10:51,863 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on Group_testCreateAndDrop,,1690229451075.2febae84cf14582e40855cc44673eaf2. after waiting 0 ms 2023-07-24 20:10:51,863 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region Group_testCreateAndDrop,,1690229451075.2febae84cf14582e40855cc44673eaf2. 2023-07-24 20:10:51,868 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCreateAndDrop/2febae84cf14582e40855cc44673eaf2/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-24 20:10:51,868 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed Group_testCreateAndDrop,,1690229451075.2febae84cf14582e40855cc44673eaf2. 2023-07-24 20:10:51,868 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 2febae84cf14582e40855cc44673eaf2: 2023-07-24 20:10:51,870 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 2febae84cf14582e40855cc44673eaf2 2023-07-24 20:10:51,870 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=2febae84cf14582e40855cc44673eaf2, regionState=CLOSED 2023-07-24 20:10:51,871 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"Group_testCreateAndDrop,,1690229451075.2febae84cf14582e40855cc44673eaf2.","families":{"info":[{"qualifier":"regioninfo","vlen":57,"tag":[],"timestamp":"1690229451870"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229451870"}]},"ts":"1690229451870"} 2023-07-24 20:10:51,874 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=81, resume processing ppid=80 2023-07-24 20:10:51,874 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=80, state=SUCCESS; CloseRegionProcedure 2febae84cf14582e40855cc44673eaf2, server=jenkins-hbase4.apache.org,45209,1690229445078 in 162 msec 2023-07-24 20:10:51,875 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=80, resume processing ppid=79 2023-07-24 20:10:51,876 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=Group_testCreateAndDrop, region=2febae84cf14582e40855cc44673eaf2, UNASSIGN in 169 msec 2023-07-24 20:10:51,876 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCreateAndDrop","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690229451876"}]},"ts":"1690229451876"} 2023-07-24 20:10:51,878 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCreateAndDrop, state=DISABLED in hbase:meta 2023-07-24 20:10:51,879 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set Group_testCreateAndDrop to state=DISABLED 2023-07-24 20:10:51,881 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=Group_testCreateAndDrop in 183 msec 2023-07-24 20:10:52,003 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-07-24 20:10:52,003 INFO [Listener at localhost/44145] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:Group_testCreateAndDrop, procId: 79 completed 2023-07-24 20:10:52,004 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete Group_testCreateAndDrop 2023-07-24 20:10:52,004 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] procedure2.ProcedureExecutor(1029): Stored pid=82, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=Group_testCreateAndDrop 2023-07-24 20:10:52,006 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=82, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=Group_testCreateAndDrop 2023-07-24 20:10:52,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint(577): Removing deleted table 'Group_testCreateAndDrop' from rsgroup 'default' 2023-07-24 20:10:52,007 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=82, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=Group_testCreateAndDrop 2023-07-24 20:10:52,009 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:52,009 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:52,010 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-24 20:10:52,012 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateAndDrop/2febae84cf14582e40855cc44673eaf2 2023-07-24 20:10:52,013 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateAndDrop/2febae84cf14582e40855cc44673eaf2/cf, FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateAndDrop/2febae84cf14582e40855cc44673eaf2/recovered.edits] 2023-07-24 20:10:52,017 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=82 2023-07-24 20:10:52,019 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateAndDrop/2febae84cf14582e40855cc44673eaf2/recovered.edits/4.seqid to hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/archive/data/default/Group_testCreateAndDrop/2febae84cf14582e40855cc44673eaf2/recovered.edits/4.seqid 2023-07-24 20:10:52,020 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCreateAndDrop/2febae84cf14582e40855cc44673eaf2 2023-07-24 20:10:52,020 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived Group_testCreateAndDrop regions 2023-07-24 20:10:52,023 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=82, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=Group_testCreateAndDrop 2023-07-24 20:10:52,025 WARN [PEWorker-3] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of Group_testCreateAndDrop from hbase:meta 2023-07-24 20:10:52,027 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(421): Removing 'Group_testCreateAndDrop' descriptor. 2023-07-24 20:10:52,028 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=82, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=Group_testCreateAndDrop 2023-07-24 20:10:52,028 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(411): Removing 'Group_testCreateAndDrop' from region states. 2023-07-24 20:10:52,028 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCreateAndDrop,,1690229451075.2febae84cf14582e40855cc44673eaf2.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690229452028"}]},"ts":"9223372036854775807"} 2023-07-24 20:10:52,030 INFO [PEWorker-3] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-07-24 20:10:52,030 DEBUG [PEWorker-3] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 2febae84cf14582e40855cc44673eaf2, NAME => 'Group_testCreateAndDrop,,1690229451075.2febae84cf14582e40855cc44673eaf2.', STARTKEY => '', ENDKEY => ''}] 2023-07-24 20:10:52,030 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(415): Marking 'Group_testCreateAndDrop' as deleted. 2023-07-24 20:10:52,030 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCreateAndDrop","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1690229452030"}]},"ts":"9223372036854775807"} 2023-07-24 20:10:52,032 INFO [PEWorker-3] hbase.MetaTableAccessor(1658): Deleted table Group_testCreateAndDrop state from META 2023-07-24 20:10:52,034 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(130): Finished pid=82, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=Group_testCreateAndDrop 2023-07-24 20:10:52,035 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=82, state=SUCCESS; DeleteTableProcedure table=Group_testCreateAndDrop in 30 msec 2023-07-24 20:10:52,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=82 2023-07-24 20:10:52,118 INFO [Listener at localhost/44145] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:Group_testCreateAndDrop, procId: 82 completed 2023-07-24 20:10:52,122 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:52,123 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:52,124 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-24 20:10:52,124 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-24 20:10:52,124 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-24 20:10:52,125 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-24 20:10:52,125 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-24 20:10:52,126 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-24 20:10:52,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:52,130 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-24 20:10:52,132 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-24 20:10:52,136 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase(152): Restoring servers: 0 2023-07-24 20:10:52,137 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-24 20:10:52,139 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:52,139 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:52,140 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-24 20:10:52,143 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-24 20:10:52,145 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:52,145 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:52,147 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:39247] to rsgroup master 2023-07-24 20:10:52,147 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-24 20:10:52,148 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.CallRunner(144): callId: 455 service: MasterService methodName: ExecMasterService size: 119 connection: 172.31.14.131:36610 deadline: 1690230652147, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. 2023-07-24 20:10:52,148 WARN [Listener at localhost/44145] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.afterMethod(TestRSGroupsBasics.java:82) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-24 20:10:52,152 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:10:52,153 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:52,153 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:52,153 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:34483, jenkins-hbase4.apache.org:34917, jenkins-hbase4.apache.org:39791, jenkins-hbase4.apache.org:45209], Tables:[hbase:meta, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-24 20:10:52,154 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-24 20:10:52,154 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-24 20:10:52,173 INFO [Listener at localhost/44145] hbase.ResourceChecker(175): after: rsgroup.TestRSGroupsBasics#testCreateAndDrop Thread=508 (was 505) Potentially hanging thread: hconnection-0x4636fd6e-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xb35bd70-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/cluster_82e812c0-1679-84b5-414c-93cfc125cc3f/dfs/data/data6/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/cluster_82e812c0-1679-84b5-414c-93cfc125cc3f/dfs/data/data5/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_2059508162_17 at /127.0.0.1:44430 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=792 (was 792), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=426 (was 426), ProcessCount=176 (was 176), AvailableMemoryMB=5747 (was 5830) 2023-07-24 20:10:52,173 WARN [Listener at localhost/44145] hbase.ResourceChecker(130): Thread=508 is superior to 500 2023-07-24 20:10:52,190 INFO [Listener at localhost/44145] hbase.ResourceChecker(147): before: rsgroup.TestRSGroupsBasics#testCloneSnapshot Thread=508, OpenFileDescriptor=792, MaxFileDescriptor=60000, SystemLoadAverage=426, ProcessCount=176, AvailableMemoryMB=5746 2023-07-24 20:10:52,190 WARN [Listener at localhost/44145] hbase.ResourceChecker(130): Thread=508 is superior to 500 2023-07-24 20:10:52,190 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase(132): testCloneSnapshot 2023-07-24 20:10:52,195 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:52,195 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:52,196 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-24 20:10:52,196 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-24 20:10:52,196 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-24 20:10:52,197 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-24 20:10:52,197 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-24 20:10:52,198 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-24 20:10:52,202 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:52,203 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-24 20:10:52,204 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-24 20:10:52,207 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase(152): Restoring servers: 0 2023-07-24 20:10:52,208 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-24 20:10:52,211 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:52,211 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:52,212 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-24 20:10:52,214 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-24 20:10:52,216 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:52,217 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:52,218 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:39247] to rsgroup master 2023-07-24 20:10:52,219 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-24 20:10:52,219 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.CallRunner(144): callId: 483 service: MasterService methodName: ExecMasterService size: 119 connection: 172.31.14.131:36610 deadline: 1690230652218, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. 2023-07-24 20:10:52,219 WARN [Listener at localhost/44145] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.setUpBeforeMethod(TestRSGroupsBase.java:136) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.beforeMethod(TestRSGroupsBasics.java:77) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-24 20:10:52,221 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:10:52,221 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:52,221 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:52,221 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:34483, jenkins-hbase4.apache.org:34917, jenkins-hbase4.apache.org:39791, jenkins-hbase4.apache.org:45209], Tables:[hbase:meta, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-24 20:10:52,222 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-24 20:10:52,222 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-24 20:10:52,224 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'Group_testCloneSnapshot', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '1'}}, {NAME => 'test', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-24 20:10:52,225 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] procedure2.ProcedureExecutor(1029): Stored pid=83, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=Group_testCloneSnapshot 2023-07-24 20:10:52,227 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=83, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=Group_testCloneSnapshot execute state=CREATE_TABLE_PRE_OPERATION 2023-07-24 20:10:52,227 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "Group_testCloneSnapshot" procId is: 83 2023-07-24 20:10:52,228 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=83 2023-07-24 20:10:52,229 DEBUG [PEWorker-2] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:52,229 DEBUG [PEWorker-2] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:52,230 DEBUG [PEWorker-2] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-24 20:10:52,232 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=83, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=Group_testCloneSnapshot execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-24 20:10:52,234 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCloneSnapshot/1078e23774518fc7d9766e7e01f10862 2023-07-24 20:10:52,235 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCloneSnapshot/1078e23774518fc7d9766e7e01f10862 empty. 2023-07-24 20:10:52,235 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCloneSnapshot/1078e23774518fc7d9766e7e01f10862 2023-07-24 20:10:52,235 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived Group_testCloneSnapshot regions 2023-07-24 20:10:52,250 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCloneSnapshot/.tabledesc/.tableinfo.0000000001 2023-07-24 20:10:52,251 INFO [RegionOpenAndInit-Group_testCloneSnapshot-pool-0] regionserver.HRegion(7675): creating {ENCODED => 1078e23774518fc7d9766e7e01f10862, NAME => 'Group_testCloneSnapshot,,1690229452224.1078e23774518fc7d9766e7e01f10862.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='Group_testCloneSnapshot', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '1'}}, {NAME => 'test', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp 2023-07-24 20:10:52,263 DEBUG [RegionOpenAndInit-Group_testCloneSnapshot-pool-0] regionserver.HRegion(866): Instantiated Group_testCloneSnapshot,,1690229452224.1078e23774518fc7d9766e7e01f10862.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:52,263 DEBUG [RegionOpenAndInit-Group_testCloneSnapshot-pool-0] regionserver.HRegion(1604): Closing 1078e23774518fc7d9766e7e01f10862, disabling compactions & flushes 2023-07-24 20:10:52,263 INFO [RegionOpenAndInit-Group_testCloneSnapshot-pool-0] regionserver.HRegion(1626): Closing region Group_testCloneSnapshot,,1690229452224.1078e23774518fc7d9766e7e01f10862. 2023-07-24 20:10:52,263 DEBUG [RegionOpenAndInit-Group_testCloneSnapshot-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCloneSnapshot,,1690229452224.1078e23774518fc7d9766e7e01f10862. 2023-07-24 20:10:52,264 DEBUG [RegionOpenAndInit-Group_testCloneSnapshot-pool-0] regionserver.HRegion(1714): Acquired close lock on Group_testCloneSnapshot,,1690229452224.1078e23774518fc7d9766e7e01f10862. after waiting 0 ms 2023-07-24 20:10:52,264 DEBUG [RegionOpenAndInit-Group_testCloneSnapshot-pool-0] regionserver.HRegion(1724): Updates disabled for region Group_testCloneSnapshot,,1690229452224.1078e23774518fc7d9766e7e01f10862. 2023-07-24 20:10:52,264 INFO [RegionOpenAndInit-Group_testCloneSnapshot-pool-0] regionserver.HRegion(1838): Closed Group_testCloneSnapshot,,1690229452224.1078e23774518fc7d9766e7e01f10862. 2023-07-24 20:10:52,264 DEBUG [RegionOpenAndInit-Group_testCloneSnapshot-pool-0] regionserver.HRegion(1558): Region close journal for 1078e23774518fc7d9766e7e01f10862: 2023-07-24 20:10:52,266 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=83, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=Group_testCloneSnapshot execute state=CREATE_TABLE_ADD_TO_META 2023-07-24 20:10:52,267 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"Group_testCloneSnapshot,,1690229452224.1078e23774518fc7d9766e7e01f10862.","families":{"info":[{"qualifier":"regioninfo","vlen":57,"tag":[],"timestamp":"1690229452267"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229452267"}]},"ts":"1690229452267"} 2023-07-24 20:10:52,269 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-24 20:10:52,270 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=83, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=Group_testCloneSnapshot execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-24 20:10:52,270 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCloneSnapshot","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690229452270"}]},"ts":"1690229452270"} 2023-07-24 20:10:52,271 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCloneSnapshot, state=ENABLING in hbase:meta 2023-07-24 20:10:52,274 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-24 20:10:52,274 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 20:10:52,274 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 20:10:52,274 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 20:10:52,274 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-24 20:10:52,274 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 20:10:52,275 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=84, ppid=83, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCloneSnapshot, region=1078e23774518fc7d9766e7e01f10862, ASSIGN}] 2023-07-24 20:10:52,276 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=84, ppid=83, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCloneSnapshot, region=1078e23774518fc7d9766e7e01f10862, ASSIGN 2023-07-24 20:10:52,277 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=84, ppid=83, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=Group_testCloneSnapshot, region=1078e23774518fc7d9766e7e01f10862, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45209,1690229445078; forceNewPlan=false, retain=false 2023-07-24 20:10:52,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=83 2023-07-24 20:10:52,428 INFO [jenkins-hbase4:39247] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-24 20:10:52,429 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=84 updating hbase:meta row=1078e23774518fc7d9766e7e01f10862, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:52,429 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCloneSnapshot,,1690229452224.1078e23774518fc7d9766e7e01f10862.","families":{"info":[{"qualifier":"regioninfo","vlen":57,"tag":[],"timestamp":"1690229452429"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229452429"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229452429"}]},"ts":"1690229452429"} 2023-07-24 20:10:52,431 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=85, ppid=84, state=RUNNABLE; OpenRegionProcedure 1078e23774518fc7d9766e7e01f10862, server=jenkins-hbase4.apache.org,45209,1690229445078}] 2023-07-24 20:10:52,530 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=83 2023-07-24 20:10:52,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open Group_testCloneSnapshot,,1690229452224.1078e23774518fc7d9766e7e01f10862. 2023-07-24 20:10:52,587 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1078e23774518fc7d9766e7e01f10862, NAME => 'Group_testCloneSnapshot,,1690229452224.1078e23774518fc7d9766e7e01f10862.', STARTKEY => '', ENDKEY => ''} 2023-07-24 20:10:52,587 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table Group_testCloneSnapshot 1078e23774518fc7d9766e7e01f10862 2023-07-24 20:10:52,587 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated Group_testCloneSnapshot,,1690229452224.1078e23774518fc7d9766e7e01f10862.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:52,587 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1078e23774518fc7d9766e7e01f10862 2023-07-24 20:10:52,587 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1078e23774518fc7d9766e7e01f10862 2023-07-24 20:10:52,589 INFO [StoreOpener-1078e23774518fc7d9766e7e01f10862-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family test of region 1078e23774518fc7d9766e7e01f10862 2023-07-24 20:10:52,591 DEBUG [StoreOpener-1078e23774518fc7d9766e7e01f10862-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCloneSnapshot/1078e23774518fc7d9766e7e01f10862/test 2023-07-24 20:10:52,591 DEBUG [StoreOpener-1078e23774518fc7d9766e7e01f10862-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCloneSnapshot/1078e23774518fc7d9766e7e01f10862/test 2023-07-24 20:10:52,591 INFO [StoreOpener-1078e23774518fc7d9766e7e01f10862-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1078e23774518fc7d9766e7e01f10862 columnFamilyName test 2023-07-24 20:10:52,592 INFO [StoreOpener-1078e23774518fc7d9766e7e01f10862-1] regionserver.HStore(310): Store=1078e23774518fc7d9766e7e01f10862/test, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:10:52,592 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCloneSnapshot/1078e23774518fc7d9766e7e01f10862 2023-07-24 20:10:52,593 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCloneSnapshot/1078e23774518fc7d9766e7e01f10862 2023-07-24 20:10:52,596 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1078e23774518fc7d9766e7e01f10862 2023-07-24 20:10:52,598 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCloneSnapshot/1078e23774518fc7d9766e7e01f10862/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 20:10:52,598 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1078e23774518fc7d9766e7e01f10862; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=10797999200, jitterRate=0.005642041563987732}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 20:10:52,599 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1078e23774518fc7d9766e7e01f10862: 2023-07-24 20:10:52,599 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for Group_testCloneSnapshot,,1690229452224.1078e23774518fc7d9766e7e01f10862., pid=85, masterSystemTime=1690229452583 2023-07-24 20:10:52,601 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for Group_testCloneSnapshot,,1690229452224.1078e23774518fc7d9766e7e01f10862. 2023-07-24 20:10:52,601 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened Group_testCloneSnapshot,,1690229452224.1078e23774518fc7d9766e7e01f10862. 2023-07-24 20:10:52,601 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=84 updating hbase:meta row=1078e23774518fc7d9766e7e01f10862, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:52,601 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"Group_testCloneSnapshot,,1690229452224.1078e23774518fc7d9766e7e01f10862.","families":{"info":[{"qualifier":"regioninfo","vlen":57,"tag":[],"timestamp":"1690229452601"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690229452601"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690229452601"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690229452601"}]},"ts":"1690229452601"} 2023-07-24 20:10:52,604 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=85, resume processing ppid=84 2023-07-24 20:10:52,604 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=85, ppid=84, state=SUCCESS; OpenRegionProcedure 1078e23774518fc7d9766e7e01f10862, server=jenkins-hbase4.apache.org,45209,1690229445078 in 172 msec 2023-07-24 20:10:52,606 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=84, resume processing ppid=83 2023-07-24 20:10:52,606 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=84, ppid=83, state=SUCCESS; TransitRegionStateProcedure table=Group_testCloneSnapshot, region=1078e23774518fc7d9766e7e01f10862, ASSIGN in 330 msec 2023-07-24 20:10:52,607 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=83, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=Group_testCloneSnapshot execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-24 20:10:52,607 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCloneSnapshot","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690229452607"}]},"ts":"1690229452607"} 2023-07-24 20:10:52,608 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCloneSnapshot, state=ENABLED in hbase:meta 2023-07-24 20:10:52,611 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=83, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=Group_testCloneSnapshot execute state=CREATE_TABLE_POST_OPERATION 2023-07-24 20:10:52,612 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=83, state=SUCCESS; CreateTableProcedure table=Group_testCloneSnapshot in 387 msec 2023-07-24 20:10:52,832 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=83 2023-07-24 20:10:52,832 INFO [Listener at localhost/44145] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:Group_testCloneSnapshot, procId: 83 completed 2023-07-24 20:10:52,832 DEBUG [Listener at localhost/44145] hbase.HBaseTestingUtility(3430): Waiting until all regions of table Group_testCloneSnapshot get assigned. Timeout = 60000ms 2023-07-24 20:10:52,832 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:10:52,837 INFO [Listener at localhost/44145] hbase.HBaseTestingUtility(3484): All regions for table Group_testCloneSnapshot assigned to meta. Checking AM states. 2023-07-24 20:10:52,838 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:10:52,838 INFO [Listener at localhost/44145] hbase.HBaseTestingUtility(3504): All regions for table Group_testCloneSnapshot assigned. 2023-07-24 20:10:52,851 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1583): Client=jenkins//172.31.14.131 snapshot request for:{ ss=Group_testCloneSnapshot_snap table=Group_testCloneSnapshot type=FLUSH ttl=0 } 2023-07-24 20:10:52,851 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1690229452851 (current time:1690229452851). 2023-07-24 20:10:52,851 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-07-24 20:10:52,851 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] zookeeper.ReadOnlyZKClient(139): Connect 0x58e64894 to 127.0.0.1:62881 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-24 20:10:52,863 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@361f040, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-24 20:10:52,868 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-24 20:10:52,869 INFO [RS-EventLoopGroup-7-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50072, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-24 20:10:52,870 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x58e64894 to 127.0.0.1:62881 2023-07-24 20:10:52,871 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:10:52,871 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-07-24 20:10:52,873 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] snapshot.SnapshotManager(648): Table enabled, starting distributed snapshots for { ss=Group_testCloneSnapshot_snap table=Group_testCloneSnapshot type=FLUSH ttl=0 } 2023-07-24 20:10:52,905 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] procedure2.ProcedureExecutor(1029): Stored pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=Group_testCloneSnapshot, type=EXCLUSIVE 2023-07-24 20:10:52,906 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=Group_testCloneSnapshot, type=EXCLUSIVE 2023-07-24 20:10:52,907 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=86, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=Group_testCloneSnapshot, type=EXCLUSIVE; timeout=600000, timestamp=1690230052907 2023-07-24 20:10:52,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] snapshot.SnapshotManager(653): Started snapshot: { ss=Group_testCloneSnapshot_snap table=Group_testCloneSnapshot type=FLUSH ttl=0 } 2023-07-24 20:10:52,908 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running FLUSH table snapshot Group_testCloneSnapshot_snap C_M_SNAPSHOT_TABLE on table Group_testCloneSnapshot 2023-07-24 20:10:52,910 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure2.ProcedureExecutor(1029): Stored pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=Group_testCloneSnapshot, type=SHARED 2023-07-24 20:10:52,911 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=86, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=Group_testCloneSnapshot, type=EXCLUSIVE 2023-07-24 20:10:52,914 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=Group_testCloneSnapshot, type=EXCLUSIVE in 17 msec 2023-07-24 20:10:52,914 DEBUG [Listener at localhost/44145] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=Group_testCloneSnapshot_snap table=Group_testCloneSnapshot type=FLUSH ttl=0 }'' to complete. (max 20000 ms per retry) 2023-07-24 20:10:52,915 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=Group_testCloneSnapshot, type=SHARED 2023-07-24 20:10:52,915 DEBUG [Listener at localhost/44145] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-07-24 20:10:52,916 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=87, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=Group_testCloneSnapshot, type=SHARED; timeout=600000, timestamp=1690230052916 2023-07-24 20:10:53,015 DEBUG [Listener at localhost/44145] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-07-24 20:10:53,018 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=Group_testCloneSnapshot_snap table=Group_testCloneSnapshot type=FLUSH ttl=0 } is done 2023-07-24 20:10:53,020 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] snapshot.SnapshotManager(404): Snapshoting '{ ss=Group_testCloneSnapshot_snap table=Group_testCloneSnapshot type=FLUSH ttl=0 }' is still in progress! 2023-07-24 20:10:53,021 DEBUG [Listener at localhost/44145] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-07-24 20:10:53,221 DEBUG [Listener at localhost/44145] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-07-24 20:10:53,222 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=Group_testCloneSnapshot_snap table=Group_testCloneSnapshot type=FLUSH ttl=0 } is done 2023-07-24 20:10:53,222 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] snapshot.SnapshotManager(404): Snapshoting '{ ss=Group_testCloneSnapshot_snap table=Group_testCloneSnapshot type=FLUSH ttl=0 }' is still in progress! 2023-07-24 20:10:53,222 DEBUG [Listener at localhost/44145] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-07-24 20:10:53,345 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure.ProcedureCoordinator(165): Submitting procedure Group_testCloneSnapshot_snap 2023-07-24 20:10:53,345 INFO [(jenkins-hbase4.apache.org,39247,1690229437650)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'Group_testCloneSnapshot_snap' 2023-07-24 20:10:53,345 DEBUG [(jenkins-hbase4.apache.org,39247,1690229437650)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-07-24 20:10:53,345 DEBUG [(jenkins-hbase4.apache.org,39247,1690229437650)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'Group_testCloneSnapshot_snap' starting 'acquire' 2023-07-24 20:10:53,345 DEBUG [(jenkins-hbase4.apache.org,39247,1690229437650)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'Group_testCloneSnapshot_snap', kicking off acquire phase on members. 2023-07-24 20:10:53,346 DEBUG [(jenkins-hbase4.apache.org,39247,1690229437650)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,346 DEBUG [(jenkins-hbase4.apache.org,39247,1690229437650)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,347 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-07-24 20:10:53,347 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34917-0x10198f4f3ce000b, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-07-24 20:10:53,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-07-24 20:10:53,347 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:45209-0x10198f4f3ce000d, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-07-24 20:10:53,347 DEBUG [(jenkins-hbase4.apache.org,39247,1690229437650)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:53,347 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-07-24 20:10:53,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-07-24 20:10:53,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-24 20:10:53,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-07-24 20:10:53,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-24 20:10:53,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-07-24 20:10:53,348 DEBUG [(jenkins-hbase4.apache.org,39247,1690229437650)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:53,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-24 20:10:53,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-24 20:10:53,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,348 DEBUG [(jenkins-hbase4.apache.org,39247,1690229437650)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 20:10:53,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,348 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,348 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45209-0x10198f4f3ce000d, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,349 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 72 2023-07-24 20:10:53,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 72 2023-07-24 20:10:53,349 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34917-0x10198f4f3ce000b, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 72 2023-07-24 20:10:53,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,349 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot Group_testCloneSnapshot_snap from table Group_testCloneSnapshot type FLUSH 2023-07-24 20:10:53,349 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot Group_testCloneSnapshot_snap from table Group_testCloneSnapshot type FLUSH 2023-07-24 20:10:53,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 72 2023-07-24 20:10:53,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,349 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot Group_testCloneSnapshot_snap from table Group_testCloneSnapshot type FLUSH 2023-07-24 20:10:53,349 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot Group_testCloneSnapshot_snap from table Group_testCloneSnapshot type FLUSH 2023-07-24 20:10:53,352 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:Group_testCloneSnapshot_snap 2023-07-24 20:10:53,352 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:Group_testCloneSnapshot_snap 2023-07-24 20:10:53,353 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:Group_testCloneSnapshot_snap 2023-07-24 20:10:53,353 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:Group_testCloneSnapshot_snap 2023-07-24 20:10:53,353 DEBUG [member: 'jenkins-hbase4.apache.org,45209,1690229445078' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'Group_testCloneSnapshot_snap' with timeout 300000ms 2023-07-24 20:10:53,353 DEBUG [member: 'jenkins-hbase4.apache.org,34917,1690229443609' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'Group_testCloneSnapshot_snap' with timeout 300000ms 2023-07-24 20:10:53,353 DEBUG [member: 'jenkins-hbase4.apache.org,39791,1690229440084' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'Group_testCloneSnapshot_snap' with timeout 300000ms 2023-07-24 20:10:53,353 DEBUG [member: 'jenkins-hbase4.apache.org,34483,1690229439858' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'Group_testCloneSnapshot_snap' with timeout 300000ms 2023-07-24 20:10:53,353 DEBUG [member: 'jenkins-hbase4.apache.org,34483,1690229439858' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-07-24 20:10:53,353 DEBUG [member: 'jenkins-hbase4.apache.org,39791,1690229440084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-07-24 20:10:53,353 DEBUG [member: 'jenkins-hbase4.apache.org,34917,1690229443609' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-07-24 20:10:53,354 DEBUG [member: 'jenkins-hbase4.apache.org,39791,1690229440084' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'Group_testCloneSnapshot_snap' starting 'acquire' stage 2023-07-24 20:10:53,355 DEBUG [member: 'jenkins-hbase4.apache.org,34917,1690229443609' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'Group_testCloneSnapshot_snap' starting 'acquire' stage 2023-07-24 20:10:53,355 DEBUG [member: 'jenkins-hbase4.apache.org,34917,1690229443609' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'Group_testCloneSnapshot_snap' locally acquired 2023-07-24 20:10:53,353 DEBUG [member: 'jenkins-hbase4.apache.org,45209,1690229445078' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-07-24 20:10:53,355 DEBUG [member: 'jenkins-hbase4.apache.org,34917,1690229443609' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34917,1690229443609' joining acquired barrier for procedure (Group_testCloneSnapshot_snap) in zk 2023-07-24 20:10:53,355 DEBUG [member: 'jenkins-hbase4.apache.org,45209,1690229445078' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'Group_testCloneSnapshot_snap' starting 'acquire' stage 2023-07-24 20:10:53,355 DEBUG [member: 'jenkins-hbase4.apache.org,45209,1690229445078' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'Group_testCloneSnapshot_snap' locally acquired 2023-07-24 20:10:53,355 DEBUG [member: 'jenkins-hbase4.apache.org,39791,1690229440084' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'Group_testCloneSnapshot_snap' locally acquired 2023-07-24 20:10:53,356 DEBUG [member: 'jenkins-hbase4.apache.org,39791,1690229440084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,39791,1690229440084' joining acquired barrier for procedure (Group_testCloneSnapshot_snap) in zk 2023-07-24 20:10:53,354 DEBUG [member: 'jenkins-hbase4.apache.org,34483,1690229439858' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'Group_testCloneSnapshot_snap' starting 'acquire' stage 2023-07-24 20:10:53,356 DEBUG [member: 'jenkins-hbase4.apache.org,34483,1690229439858' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'Group_testCloneSnapshot_snap' locally acquired 2023-07-24 20:10:53,356 DEBUG [member: 'jenkins-hbase4.apache.org,34483,1690229439858' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,34483,1690229439858' joining acquired barrier for procedure (Group_testCloneSnapshot_snap) in zk 2023-07-24 20:10:53,355 DEBUG [member: 'jenkins-hbase4.apache.org,45209,1690229445078' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45209,1690229445078' joining acquired barrier for procedure (Group_testCloneSnapshot_snap) in zk 2023-07-24 20:10:53,357 DEBUG [member: 'jenkins-hbase4.apache.org,34917,1690229443609' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,361 DEBUG [member: 'jenkins-hbase4.apache.org,39791,1690229440084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,361 DEBUG [member: 'jenkins-hbase4.apache.org,45209,1690229445078' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,361 DEBUG [member: 'jenkins-hbase4.apache.org,34917,1690229443609' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34917-0x10198f4f3ce000b, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,361 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:53,361 DEBUG [member: 'jenkins-hbase4.apache.org,34483,1690229439858' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,361 DEBUG [member: 'jenkins-hbase4.apache.org,34917,1690229443609' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'Group_testCloneSnapshot_snap' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 20:10:53,361 DEBUG [member: 'jenkins-hbase4.apache.org,39791,1690229440084' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,361 DEBUG [member: 'jenkins-hbase4.apache.org,39791,1690229440084' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'Group_testCloneSnapshot_snap' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 20:10:53,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:53,361 DEBUG [member: 'jenkins-hbase4.apache.org,45209,1690229445078' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45209-0x10198f4f3ce000d, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,361 DEBUG [member: 'jenkins-hbase4.apache.org,45209,1690229445078' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'Group_testCloneSnapshot_snap' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 20:10:53,361 DEBUG [member: 'jenkins-hbase4.apache.org,34483,1690229439858' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,361 DEBUG [member: 'jenkins-hbase4.apache.org,34483,1690229439858' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'Group_testCloneSnapshot_snap' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 20:10:53,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 20:10:53,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-07-24 20:10:53,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 20:10:53,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 20:10:53,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----Group_testCloneSnapshot_snap 2023-07-24 20:10:53,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:53,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:53,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:53,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:53,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 20:10:53,364 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45209,1690229445078' joining acquired barrier for procedure 'Group_testCloneSnapshot_snap' on coordinator 2023-07-24 20:10:53,364 DEBUG [(jenkins-hbase4.apache.org,39247,1690229437650)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'Group_testCloneSnapshot_snap' starting 'in-barrier' execution. 2023-07-24 20:10:53,364 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6726de48[Count = 0] remaining members to acquire global barrier 2023-07-24 20:10:53,364 DEBUG [(jenkins-hbase4.apache.org,39247,1690229437650)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,365 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:45209-0x10198f4f3ce000d, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,365 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34917-0x10198f4f3ce000b, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,365 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,365 DEBUG [member: 'jenkins-hbase4.apache.org,45209,1690229445078' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'Group_testCloneSnapshot_snap' received 'reached' from coordinator. 2023-07-24 20:10:53,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,365 DEBUG [(jenkins-hbase4.apache.org,39247,1690229437650)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:53,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,366 DEBUG [(jenkins-hbase4.apache.org,39247,1690229437650)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 20:10:53,366 DEBUG [member: 'jenkins-hbase4.apache.org,34483,1690229439858' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'Group_testCloneSnapshot_snap' received 'reached' from coordinator. 2023-07-24 20:10:53,365 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,366 DEBUG [member: 'jenkins-hbase4.apache.org,34483,1690229439858' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'Group_testCloneSnapshot_snap' locally completed 2023-07-24 20:10:53,366 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,366 DEBUG [member: 'jenkins-hbase4.apache.org,39791,1690229440084' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'Group_testCloneSnapshot_snap' received 'reached' from coordinator. 2023-07-24 20:10:53,366 DEBUG [member: 'jenkins-hbase4.apache.org,34917,1690229443609' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'Group_testCloneSnapshot_snap' received 'reached' from coordinator. 2023-07-24 20:10:53,366 DEBUG [member: 'jenkins-hbase4.apache.org,34483,1690229439858' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'Group_testCloneSnapshot_snap' completed for member 'jenkins-hbase4.apache.org,34483,1690229439858' in zk 2023-07-24 20:10:53,366 DEBUG [member: 'jenkins-hbase4.apache.org,34917,1690229443609' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'Group_testCloneSnapshot_snap' locally completed 2023-07-24 20:10:53,366 DEBUG [member: 'jenkins-hbase4.apache.org,34917,1690229443609' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'Group_testCloneSnapshot_snap' completed for member 'jenkins-hbase4.apache.org,34917,1690229443609' in zk 2023-07-24 20:10:53,366 DEBUG [member: 'jenkins-hbase4.apache.org,39791,1690229440084' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'Group_testCloneSnapshot_snap' locally completed 2023-07-24 20:10:53,366 DEBUG [member: 'jenkins-hbase4.apache.org,39791,1690229440084' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'Group_testCloneSnapshot_snap' completed for member 'jenkins-hbase4.apache.org,39791,1690229440084' in zk 2023-07-24 20:10:53,366 DEBUG [member: 'jenkins-hbase4.apache.org,45209,1690229445078' subprocedure-pool-0] snapshot.FlushSnapshotSubprocedure(170): Flush Snapshot Tasks submitted for 1 regions 2023-07-24 20:10:53,367 DEBUG [member: 'jenkins-hbase4.apache.org,45209,1690229445078' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(301): Waiting for local region snapshots to finish. 2023-07-24 20:10:53,367 DEBUG [rs(jenkins-hbase4.apache.org,45209,1690229445078)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on Group_testCloneSnapshot,,1690229452224.1078e23774518fc7d9766e7e01f10862. 2023-07-24 20:10:53,367 DEBUG [rs(jenkins-hbase4.apache.org,45209,1690229445078)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region Group_testCloneSnapshot,,1690229452224.1078e23774518fc7d9766e7e01f10862. started... 2023-07-24 20:10:53,368 DEBUG [rs(jenkins-hbase4.apache.org,45209,1690229445078)-snapshot-pool-0] regionserver.HRegion(2446): Flush status journal for 1078e23774518fc7d9766e7e01f10862: 2023-07-24 20:10:53,368 DEBUG [member: 'jenkins-hbase4.apache.org,34483,1690229439858' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'Group_testCloneSnapshot_snap' has notified controller of completion 2023-07-24 20:10:53,368 DEBUG [member: 'jenkins-hbase4.apache.org,34483,1690229439858' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 20:10:53,368 DEBUG [member: 'jenkins-hbase4.apache.org,34917,1690229443609' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'Group_testCloneSnapshot_snap' has notified controller of completion 2023-07-24 20:10:53,369 DEBUG [member: 'jenkins-hbase4.apache.org,34917,1690229443609' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 20:10:53,368 DEBUG [member: 'jenkins-hbase4.apache.org,34483,1690229439858' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'Group_testCloneSnapshot_snap' completed. 2023-07-24 20:10:53,370 DEBUG [rs(jenkins-hbase4.apache.org,45209,1690229445078)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'Group_testCloneSnapshot,,1690229452224.1078e23774518fc7d9766e7e01f10862.' region-info for snapshot=Group_testCloneSnapshot_snap 2023-07-24 20:10:53,369 DEBUG [member: 'jenkins-hbase4.apache.org,34917,1690229443609' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'Group_testCloneSnapshot_snap' completed. 2023-07-24 20:10:53,370 DEBUG [member: 'jenkins-hbase4.apache.org,39791,1690229440084' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'Group_testCloneSnapshot_snap' has notified controller of completion 2023-07-24 20:10:53,370 DEBUG [member: 'jenkins-hbase4.apache.org,39791,1690229440084' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 20:10:53,370 DEBUG [member: 'jenkins-hbase4.apache.org,39791,1690229440084' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'Group_testCloneSnapshot_snap' completed. 2023-07-24 20:10:53,376 DEBUG [rs(jenkins-hbase4.apache.org,45209,1690229445078)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-07-24 20:10:53,380 DEBUG [rs(jenkins-hbase4.apache.org,45209,1690229445078)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [] hfiles 2023-07-24 20:10:53,393 DEBUG [rs(jenkins-hbase4.apache.org,45209,1690229445078)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region Group_testCloneSnapshot,,1690229452224.1078e23774518fc7d9766e7e01f10862. completed. 2023-07-24 20:10:53,393 DEBUG [rs(jenkins-hbase4.apache.org,45209,1690229445078)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on Group_testCloneSnapshot,,1690229452224.1078e23774518fc7d9766e7e01f10862. 2023-07-24 20:10:53,394 DEBUG [member: 'jenkins-hbase4.apache.org,45209,1690229445078' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 1/1 local region snapshots. 2023-07-24 20:10:53,394 DEBUG [member: 'jenkins-hbase4.apache.org,45209,1690229445078' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(314): Completed 1 local region snapshots. 2023-07-24 20:10:53,394 DEBUG [member: 'jenkins-hbase4.apache.org,45209,1690229445078' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(345): cancelling 0 tasks for snapshot jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:53,394 DEBUG [member: 'jenkins-hbase4.apache.org,45209,1690229445078' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'Group_testCloneSnapshot_snap' locally completed 2023-07-24 20:10:53,394 DEBUG [member: 'jenkins-hbase4.apache.org,45209,1690229445078' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'Group_testCloneSnapshot_snap' completed for member 'jenkins-hbase4.apache.org,45209,1690229445078' in zk 2023-07-24 20:10:53,396 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:53,396 DEBUG [member: 'jenkins-hbase4.apache.org,45209,1690229445078' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'Group_testCloneSnapshot_snap' has notified controller of completion 2023-07-24 20:10:53,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/reached/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:53,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 20:10:53,396 DEBUG [member: 'jenkins-hbase4.apache.org,45209,1690229445078' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 20:10:53,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-07-24 20:10:53,396 DEBUG [member: 'jenkins-hbase4.apache.org,45209,1690229445078' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'Group_testCloneSnapshot_snap' completed. 2023-07-24 20:10:53,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 20:10:53,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 20:10:53,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----Group_testCloneSnapshot_snap 2023-07-24 20:10:53,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:53,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:53,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:53,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:53,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 20:10:53,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----Group_testCloneSnapshot_snap 2023-07-24 20:10:53,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:53,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:53,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:53,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:53,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'Group_testCloneSnapshot_snap' member 'jenkins-hbase4.apache.org,45209,1690229445078': 2023-07-24 20:10:53,402 INFO [(jenkins-hbase4.apache.org,39247,1690229437650)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'Group_testCloneSnapshot_snap' execution completed 2023-07-24 20:10:53,402 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45209,1690229445078' released barrier for procedure'Group_testCloneSnapshot_snap', counting down latch. Waiting for 0 more 2023-07-24 20:10:53,402 DEBUG [(jenkins-hbase4.apache.org,39247,1690229437650)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 20:10:53,402 DEBUG [(jenkins-hbase4.apache.org,39247,1690229437650)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 20:10:53,402 DEBUG [(jenkins-hbase4.apache.org,39247,1690229437650)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:Group_testCloneSnapshot_snap 2023-07-24 20:10:53,402 INFO [(jenkins-hbase4.apache.org,39247,1690229437650)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure Group_testCloneSnapshot_snapincluding nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2023-07-24 20:10:53,404 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,404 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,404 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:45209-0x10198f4f3ce000d, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,404 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:45209-0x10198f4f3ce000d, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-07-24 20:10:53,404 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-07-24 20:10:53,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 20:10:53,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-07-24 20:10:53,404 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,405 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-07-24 20:10:53,404 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34917-0x10198f4f3ce000b, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-07-24 20:10:53,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 20:10:53,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-24 20:10:53,405 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34917-0x10198f4f3ce000b, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-07-24 20:10:53,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-07-24 20:10:53,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-24 20:10:53,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----Group_testCloneSnapshot_snap 2023-07-24 20:10:53,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 20:10:53,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-07-24 20:10:53,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-24 20:10:53,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----Group_testCloneSnapshot_snap 2023-07-24 20:10:53,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-07-24 20:10:53,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-24 20:10:53,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:53,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,408 DEBUG [(jenkins-hbase4.apache.org,39247,1690229437650)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:53,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:53,408 DEBUG [(jenkins-hbase4.apache.org,39247,1690229437650)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:53,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:53,409 DEBUG [(jenkins-hbase4.apache.org,39247,1690229437650)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:53,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:53,409 DEBUG [(jenkins-hbase4.apache.org,39247,1690229437650)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:53,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 20:10:53,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----Group_testCloneSnapshot_snap 2023-07-24 20:10:53,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:53,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:53,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:53,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:53,411 DEBUG [(jenkins-hbase4.apache.org,39247,1690229437650)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:53,411 DEBUG [(jenkins-hbase4.apache.org,39247,1690229437650)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:53,412 DEBUG [(jenkins-hbase4.apache.org,39247,1690229437650)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:53,412 DEBUG [(jenkins-hbase4.apache.org,39247,1690229437650)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:53,416 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:53,416 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,416 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:53,416 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:53,416 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:53,416 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:45209-0x10198f4f3ce000d, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-07-24 20:10:53,416 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-07-24 20:10:53,416 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34917-0x10198f4f3ce000b, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-07-24 20:10:53,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-07-24 20:10:53,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-07-24 20:10:53,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-24 20:10:53,416 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-07-24 20:10:53,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-24 20:10:53,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-07-24 20:10:53,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-24 20:10:53,416 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-07-24 20:10:53,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-24 20:10:53,417 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:53,417 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,417 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:53,417 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:53,417 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:45209-0x10198f4f3ce000d, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-07-24 20:10:53,417 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-07-24 20:10:53,417 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-07-24 20:10:53,418 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34917-0x10198f4f3ce000b, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-07-24 20:10:53,418 DEBUG [(jenkins-hbase4.apache.org,39247,1690229437650)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 20:10:53,418 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(97): Done waiting - online snapshot for Group_testCloneSnapshot_snap 2023-07-24 20:10:53,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-07-24 20:10:53,418 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap/jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:53,419 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-07-24 20:10:53,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-24 20:10:53,419 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for Group_testCloneSnapshot_snap 2023-07-24 20:10:53,419 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,419 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-07-24 20:10:53,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-24 20:10:53,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-24 20:10:53,418 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-07-24 20:10:53,419 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/abort/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-24 20:10:53,422 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.hbase-snapshot/.tmp/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,458 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.hbase-snapshot/.tmp/Group_testCloneSnapshot_snap to hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.hbase-snapshot/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,492 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot Group_testCloneSnapshot_snap of table Group_testCloneSnapshot completed 2023-07-24 20:10:53,492 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.hbase-snapshot/.tmp/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,492 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.hbase-snapshot/.tmp/Group_testCloneSnapshot_snap 2023-07-24 20:10:53,493 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running FLUSH table snapshot Group_testCloneSnapshot_snap C_M_SNAPSHOT_TABLE on table Group_testCloneSnapshot at 1690229452908Consolidate snapshot: Group_testCloneSnapshot_snap at 1690229453419 (+511 ms)Loading Region manifests for Group_testCloneSnapshot_snap at 1690229453419Writing data manifest for Group_testCloneSnapshot_snap at 1690229453432 (+13 ms)Verifying snapshot: Group_testCloneSnapshot_snap at 1690229453448 (+16 ms)Snapshot Group_testCloneSnapshot_snap of table Group_testCloneSnapshot completed at 1690229453492 (+44 ms) 2023-07-24 20:10:53,494 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=87, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=Group_testCloneSnapshot, type=SHARED 2023-07-24 20:10:53,496 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=87, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=Group_testCloneSnapshot, type=SHARED in 585 msec 2023-07-24 20:10:53,523 DEBUG [Listener at localhost/44145] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-07-24 20:10:53,523 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1212): Checking to see if snapshot from request:{ ss=Group_testCloneSnapshot_snap table=Group_testCloneSnapshot type=FLUSH ttl=0 } is done 2023-07-24 20:10:53,524 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] snapshot.SnapshotManager(401): Snapshot '{ ss=Group_testCloneSnapshot_snap table=Group_testCloneSnapshot type=FLUSH ttl=0 }' has completed, notifying client. 2023-07-24 20:10:53,535 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint(486): Pre-moving table Group_testCloneSnapshot_clone to RSGroup default 2023-07-24 20:10:53,537 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:53,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:53,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-24 20:10:53,539 ERROR [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(742): TableDescriptor of table {} not found. Skipping the region movement of this table. 2023-07-24 20:10:53,554 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:CLONE_SNAPSHOT_PRE_OPERATION; CloneSnapshotProcedure (table=Group_testCloneSnapshot_clone snapshot=name: "Group_testCloneSnapshot_snap" table: "Group_testCloneSnapshot" creation_time: 1690229452851 type: FLUSH version: 2 ttl: 0 ) 2023-07-24 20:10:53,554 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] snapshot.SnapshotManager(750): Clone snapshot=Group_testCloneSnapshot_snap as table=Group_testCloneSnapshot_clone 2023-07-24 20:10:53,558 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-07-24 20:10:53,573 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCloneSnapshot_clone/.tabledesc/.tableinfo.0000000001 2023-07-24 20:10:53,578 INFO [PEWorker-4] snapshot.RestoreSnapshotHelper(177): starting restore table regions using snapshot=name: "Group_testCloneSnapshot_snap" table: "Group_testCloneSnapshot" creation_time: 1690229452851 type: FLUSH version: 2 ttl: 0 2023-07-24 20:10:53,579 DEBUG [PEWorker-4] snapshot.RestoreSnapshotHelper(785): get table regions: hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCloneSnapshot_clone 2023-07-24 20:10:53,580 INFO [PEWorker-4] snapshot.RestoreSnapshotHelper(239): region to add: 1078e23774518fc7d9766e7e01f10862 2023-07-24 20:10:53,580 INFO [PEWorker-4] snapshot.RestoreSnapshotHelper(585): clone region=1078e23774518fc7d9766e7e01f10862 as dc99354a9c892a2852932acf3304fbbc in snapshot Group_testCloneSnapshot_snap 2023-07-24 20:10:53,581 INFO [RestoreSnapshot-pool-0] regionserver.HRegion(7675): creating {ENCODED => dc99354a9c892a2852932acf3304fbbc, NAME => 'Group_testCloneSnapshot_clone,,1690229452224.dc99354a9c892a2852932acf3304fbbc.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='Group_testCloneSnapshot_clone', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '1'}}, {NAME => 'test', BLOOMFILTER => 'NONE', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp 2023-07-24 20:10:53,592 DEBUG [RestoreSnapshot-pool-0] regionserver.HRegion(866): Instantiated Group_testCloneSnapshot_clone,,1690229452224.dc99354a9c892a2852932acf3304fbbc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:53,592 DEBUG [RestoreSnapshot-pool-0] regionserver.HRegion(1604): Closing dc99354a9c892a2852932acf3304fbbc, disabling compactions & flushes 2023-07-24 20:10:53,592 INFO [RestoreSnapshot-pool-0] regionserver.HRegion(1626): Closing region Group_testCloneSnapshot_clone,,1690229452224.dc99354a9c892a2852932acf3304fbbc. 2023-07-24 20:10:53,592 DEBUG [RestoreSnapshot-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCloneSnapshot_clone,,1690229452224.dc99354a9c892a2852932acf3304fbbc. 2023-07-24 20:10:53,592 DEBUG [RestoreSnapshot-pool-0] regionserver.HRegion(1714): Acquired close lock on Group_testCloneSnapshot_clone,,1690229452224.dc99354a9c892a2852932acf3304fbbc. after waiting 0 ms 2023-07-24 20:10:53,592 DEBUG [RestoreSnapshot-pool-0] regionserver.HRegion(1724): Updates disabled for region Group_testCloneSnapshot_clone,,1690229452224.dc99354a9c892a2852932acf3304fbbc. 2023-07-24 20:10:53,592 INFO [RestoreSnapshot-pool-0] regionserver.HRegion(1838): Closed Group_testCloneSnapshot_clone,,1690229452224.dc99354a9c892a2852932acf3304fbbc. 2023-07-24 20:10:53,592 DEBUG [RestoreSnapshot-pool-0] regionserver.HRegion(1558): Region close journal for dc99354a9c892a2852932acf3304fbbc: 2023-07-24 20:10:53,592 INFO [PEWorker-4] snapshot.RestoreSnapshotHelper(266): finishing restore table regions using snapshot=name: "Group_testCloneSnapshot_snap" table: "Group_testCloneSnapshot" creation_time: 1690229452851 type: FLUSH version: 2 ttl: 0 2023-07-24 20:10:53,592 INFO [PEWorker-4] procedure.CloneSnapshotProcedure$1(421): Clone snapshot=Group_testCloneSnapshot_snap on table=Group_testCloneSnapshot_clone completed! 2023-07-24 20:10:53,596 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"Group_testCloneSnapshot_clone,,1690229452224.dc99354a9c892a2852932acf3304fbbc.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1690229453596"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229453596"}]},"ts":"1690229453596"} 2023-07-24 20:10:53,598 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-24 20:10:53,598 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCloneSnapshot_clone","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690229453598"}]},"ts":"1690229453598"} 2023-07-24 20:10:53,599 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCloneSnapshot_clone, state=ENABLING in hbase:meta 2023-07-24 20:10:53,603 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-24 20:10:53,603 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 20:10:53,603 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 20:10:53,603 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 20:10:53,603 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(362): server 3 is on host 0 2023-07-24 20:10:53,603 DEBUG [PEWorker-4] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 20:10:53,603 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCloneSnapshot_clone, region=dc99354a9c892a2852932acf3304fbbc, ASSIGN}] 2023-07-24 20:10:53,605 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_testCloneSnapshot_clone, region=dc99354a9c892a2852932acf3304fbbc, ASSIGN 2023-07-24 20:10:53,606 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=Group_testCloneSnapshot_clone, region=dc99354a9c892a2852932acf3304fbbc, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34917,1690229443609; forceNewPlan=false, retain=false 2023-07-24 20:10:53,607 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-24 20:10:53,659 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-07-24 20:10:53,756 INFO [jenkins-hbase4:39247] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-24 20:10:53,758 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=dc99354a9c892a2852932acf3304fbbc, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:53,758 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCloneSnapshot_clone,,1690229452224.dc99354a9c892a2852932acf3304fbbc.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1690229453758"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229453758"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229453758"}]},"ts":"1690229453758"} 2023-07-24 20:10:53,760 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE; OpenRegionProcedure dc99354a9c892a2852932acf3304fbbc, server=jenkins-hbase4.apache.org,34917,1690229443609}] 2023-07-24 20:10:53,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-07-24 20:10:53,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open Group_testCloneSnapshot_clone,,1690229452224.dc99354a9c892a2852932acf3304fbbc. 2023-07-24 20:10:53,916 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => dc99354a9c892a2852932acf3304fbbc, NAME => 'Group_testCloneSnapshot_clone,,1690229452224.dc99354a9c892a2852932acf3304fbbc.', STARTKEY => '', ENDKEY => ''} 2023-07-24 20:10:53,916 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table Group_testCloneSnapshot_clone dc99354a9c892a2852932acf3304fbbc 2023-07-24 20:10:53,916 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated Group_testCloneSnapshot_clone,,1690229452224.dc99354a9c892a2852932acf3304fbbc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:53,916 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for dc99354a9c892a2852932acf3304fbbc 2023-07-24 20:10:53,916 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for dc99354a9c892a2852932acf3304fbbc 2023-07-24 20:10:53,918 INFO [StoreOpener-dc99354a9c892a2852932acf3304fbbc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family test of region dc99354a9c892a2852932acf3304fbbc 2023-07-24 20:10:53,919 DEBUG [StoreOpener-dc99354a9c892a2852932acf3304fbbc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCloneSnapshot_clone/dc99354a9c892a2852932acf3304fbbc/test 2023-07-24 20:10:53,920 DEBUG [StoreOpener-dc99354a9c892a2852932acf3304fbbc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCloneSnapshot_clone/dc99354a9c892a2852932acf3304fbbc/test 2023-07-24 20:10:53,920 INFO [StoreOpener-dc99354a9c892a2852932acf3304fbbc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region dc99354a9c892a2852932acf3304fbbc columnFamilyName test 2023-07-24 20:10:53,921 INFO [StoreOpener-dc99354a9c892a2852932acf3304fbbc-1] regionserver.HStore(310): Store=dc99354a9c892a2852932acf3304fbbc/test, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:10:53,922 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCloneSnapshot_clone/dc99354a9c892a2852932acf3304fbbc 2023-07-24 20:10:53,922 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCloneSnapshot_clone/dc99354a9c892a2852932acf3304fbbc 2023-07-24 20:10:53,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for dc99354a9c892a2852932acf3304fbbc 2023-07-24 20:10:53,927 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCloneSnapshot_clone/dc99354a9c892a2852932acf3304fbbc/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 20:10:53,928 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened dc99354a9c892a2852932acf3304fbbc; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=10401754560, jitterRate=-0.03126111626625061}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 20:10:53,928 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for dc99354a9c892a2852932acf3304fbbc: 2023-07-24 20:10:53,929 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for Group_testCloneSnapshot_clone,,1690229452224.dc99354a9c892a2852932acf3304fbbc., pid=90, masterSystemTime=1690229453911 2023-07-24 20:10:53,930 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for Group_testCloneSnapshot_clone,,1690229452224.dc99354a9c892a2852932acf3304fbbc. 2023-07-24 20:10:53,930 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened Group_testCloneSnapshot_clone,,1690229452224.dc99354a9c892a2852932acf3304fbbc. 2023-07-24 20:10:53,931 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=dc99354a9c892a2852932acf3304fbbc, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:53,931 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"Group_testCloneSnapshot_clone,,1690229452224.dc99354a9c892a2852932acf3304fbbc.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1690229453931"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690229453931"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690229453931"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690229453931"}]},"ts":"1690229453931"} 2023-07-24 20:10:53,934 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-07-24 20:10:53,934 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; OpenRegionProcedure dc99354a9c892a2852932acf3304fbbc, server=jenkins-hbase4.apache.org,34917,1690229443609 in 172 msec 2023-07-24 20:10:53,935 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-07-24 20:10:53,935 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=Group_testCloneSnapshot_clone, region=dc99354a9c892a2852932acf3304fbbc, ASSIGN in 331 msec 2023-07-24 20:10:53,936 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCloneSnapshot_clone","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690229453936"}]},"ts":"1690229453936"} 2023-07-24 20:10:53,937 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCloneSnapshot_clone, state=ENABLED in hbase:meta 2023-07-24 20:10:53,943 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; CloneSnapshotProcedure (table=Group_testCloneSnapshot_clone snapshot=name: "Group_testCloneSnapshot_snap" table: "Group_testCloneSnapshot" creation_time: 1690229452851 type: FLUSH version: 2 ttl: 0 ) in 396 msec 2023-07-24 20:10:54,161 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-07-24 20:10:54,162 INFO [Listener at localhost/44145] client.HBaseAdmin$TableFuture(3541): Operation: MODIFY, Table Name: default:Group_testCloneSnapshot_clone, procId: 88 completed 2023-07-24 20:10:54,163 INFO [Listener at localhost/44145] client.HBaseAdmin$15(890): Started disable of Group_testCloneSnapshot 2023-07-24 20:10:54,164 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable Group_testCloneSnapshot 2023-07-24 20:10:54,165 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] procedure2.ProcedureExecutor(1029): Stored pid=91, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=Group_testCloneSnapshot 2023-07-24 20:10:54,167 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=91 2023-07-24 20:10:54,168 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCloneSnapshot","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690229454168"}]},"ts":"1690229454168"} 2023-07-24 20:10:54,169 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCloneSnapshot, state=DISABLING in hbase:meta 2023-07-24 20:10:54,170 INFO [PEWorker-3] procedure.DisableTableProcedure(293): Set Group_testCloneSnapshot to state=DISABLING 2023-07-24 20:10:54,171 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=91, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCloneSnapshot, region=1078e23774518fc7d9766e7e01f10862, UNASSIGN}] 2023-07-24 20:10:54,174 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=92, ppid=91, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCloneSnapshot, region=1078e23774518fc7d9766e7e01f10862, UNASSIGN 2023-07-24 20:10:54,175 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=92 updating hbase:meta row=1078e23774518fc7d9766e7e01f10862, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:54,175 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCloneSnapshot,,1690229452224.1078e23774518fc7d9766e7e01f10862.","families":{"info":[{"qualifier":"regioninfo","vlen":57,"tag":[],"timestamp":"1690229454175"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229454175"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229454175"}]},"ts":"1690229454175"} 2023-07-24 20:10:54,176 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=92, state=RUNNABLE; CloseRegionProcedure 1078e23774518fc7d9766e7e01f10862, server=jenkins-hbase4.apache.org,45209,1690229445078}] 2023-07-24 20:10:54,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=91 2023-07-24 20:10:54,329 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 1078e23774518fc7d9766e7e01f10862 2023-07-24 20:10:54,329 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1078e23774518fc7d9766e7e01f10862, disabling compactions & flushes 2023-07-24 20:10:54,329 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region Group_testCloneSnapshot,,1690229452224.1078e23774518fc7d9766e7e01f10862. 2023-07-24 20:10:54,329 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCloneSnapshot,,1690229452224.1078e23774518fc7d9766e7e01f10862. 2023-07-24 20:10:54,329 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on Group_testCloneSnapshot,,1690229452224.1078e23774518fc7d9766e7e01f10862. after waiting 0 ms 2023-07-24 20:10:54,329 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region Group_testCloneSnapshot,,1690229452224.1078e23774518fc7d9766e7e01f10862. 2023-07-24 20:10:54,333 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCloneSnapshot/1078e23774518fc7d9766e7e01f10862/recovered.edits/5.seqid, newMaxSeqId=5, maxSeqId=1 2023-07-24 20:10:54,335 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed Group_testCloneSnapshot,,1690229452224.1078e23774518fc7d9766e7e01f10862. 2023-07-24 20:10:54,335 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1078e23774518fc7d9766e7e01f10862: 2023-07-24 20:10:54,336 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 1078e23774518fc7d9766e7e01f10862 2023-07-24 20:10:54,336 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=92 updating hbase:meta row=1078e23774518fc7d9766e7e01f10862, regionState=CLOSED 2023-07-24 20:10:54,337 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"Group_testCloneSnapshot,,1690229452224.1078e23774518fc7d9766e7e01f10862.","families":{"info":[{"qualifier":"regioninfo","vlen":57,"tag":[],"timestamp":"1690229454336"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229454336"}]},"ts":"1690229454336"} 2023-07-24 20:10:54,339 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=92 2023-07-24 20:10:54,339 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=92, state=SUCCESS; CloseRegionProcedure 1078e23774518fc7d9766e7e01f10862, server=jenkins-hbase4.apache.org,45209,1690229445078 in 162 msec 2023-07-24 20:10:54,341 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=91 2023-07-24 20:10:54,341 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=91, state=SUCCESS; TransitRegionStateProcedure table=Group_testCloneSnapshot, region=1078e23774518fc7d9766e7e01f10862, UNASSIGN in 168 msec 2023-07-24 20:10:54,342 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCloneSnapshot","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690229454341"}]},"ts":"1690229454341"} 2023-07-24 20:10:54,343 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCloneSnapshot, state=DISABLED in hbase:meta 2023-07-24 20:10:54,348 INFO [PEWorker-3] procedure.DisableTableProcedure(305): Set Group_testCloneSnapshot to state=DISABLED 2023-07-24 20:10:54,349 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=91, state=SUCCESS; DisableTableProcedure table=Group_testCloneSnapshot in 184 msec 2023-07-24 20:10:54,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=91 2023-07-24 20:10:54,470 INFO [Listener at localhost/44145] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:Group_testCloneSnapshot, procId: 91 completed 2023-07-24 20:10:54,470 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete Group_testCloneSnapshot 2023-07-24 20:10:54,471 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] procedure2.ProcedureExecutor(1029): Stored pid=94, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=Group_testCloneSnapshot 2023-07-24 20:10:54,473 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=94, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=Group_testCloneSnapshot 2023-07-24 20:10:54,473 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint(577): Removing deleted table 'Group_testCloneSnapshot' from rsgroup 'default' 2023-07-24 20:10:54,474 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=94, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=Group_testCloneSnapshot 2023-07-24 20:10:54,475 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:54,476 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:54,476 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-24 20:10:54,478 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCloneSnapshot/1078e23774518fc7d9766e7e01f10862 2023-07-24 20:10:54,479 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=94 2023-07-24 20:10:54,480 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCloneSnapshot/1078e23774518fc7d9766e7e01f10862/recovered.edits, FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCloneSnapshot/1078e23774518fc7d9766e7e01f10862/test] 2023-07-24 20:10:54,486 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCloneSnapshot/1078e23774518fc7d9766e7e01f10862/recovered.edits/5.seqid to hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/archive/data/default/Group_testCloneSnapshot/1078e23774518fc7d9766e7e01f10862/recovered.edits/5.seqid 2023-07-24 20:10:54,487 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCloneSnapshot/1078e23774518fc7d9766e7e01f10862 2023-07-24 20:10:54,487 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived Group_testCloneSnapshot regions 2023-07-24 20:10:54,490 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=94, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=Group_testCloneSnapshot 2023-07-24 20:10:54,492 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of Group_testCloneSnapshot from hbase:meta 2023-07-24 20:10:54,494 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'Group_testCloneSnapshot' descriptor. 2023-07-24 20:10:54,495 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=94, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=Group_testCloneSnapshot 2023-07-24 20:10:54,495 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'Group_testCloneSnapshot' from region states. 2023-07-24 20:10:54,495 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCloneSnapshot,,1690229452224.1078e23774518fc7d9766e7e01f10862.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690229454495"}]},"ts":"9223372036854775807"} 2023-07-24 20:10:54,497 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-07-24 20:10:54,497 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 1078e23774518fc7d9766e7e01f10862, NAME => 'Group_testCloneSnapshot,,1690229452224.1078e23774518fc7d9766e7e01f10862.', STARTKEY => '', ENDKEY => ''}] 2023-07-24 20:10:54,497 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'Group_testCloneSnapshot' as deleted. 2023-07-24 20:10:54,497 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCloneSnapshot","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1690229454497"}]},"ts":"9223372036854775807"} 2023-07-24 20:10:54,499 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table Group_testCloneSnapshot state from META 2023-07-24 20:10:54,501 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=94, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=Group_testCloneSnapshot 2023-07-24 20:10:54,502 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, state=SUCCESS; DeleteTableProcedure table=Group_testCloneSnapshot in 30 msec 2023-07-24 20:10:54,581 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=94 2023-07-24 20:10:54,581 INFO [Listener at localhost/44145] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:Group_testCloneSnapshot, procId: 94 completed 2023-07-24 20:10:54,581 INFO [Listener at localhost/44145] client.HBaseAdmin$15(890): Started disable of Group_testCloneSnapshot_clone 2023-07-24 20:10:54,582 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable Group_testCloneSnapshot_clone 2023-07-24 20:10:54,583 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=Group_testCloneSnapshot_clone 2023-07-24 20:10:54,586 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=95 2023-07-24 20:10:54,586 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCloneSnapshot_clone","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690229454586"}]},"ts":"1690229454586"} 2023-07-24 20:10:54,588 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCloneSnapshot_clone, state=DISABLING in hbase:meta 2023-07-24 20:10:54,589 INFO [PEWorker-2] procedure.DisableTableProcedure(293): Set Group_testCloneSnapshot_clone to state=DISABLING 2023-07-24 20:10:54,590 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=96, ppid=95, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCloneSnapshot_clone, region=dc99354a9c892a2852932acf3304fbbc, UNASSIGN}] 2023-07-24 20:10:54,592 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=96, ppid=95, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_testCloneSnapshot_clone, region=dc99354a9c892a2852932acf3304fbbc, UNASSIGN 2023-07-24 20:10:54,593 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=96 updating hbase:meta row=dc99354a9c892a2852932acf3304fbbc, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:54,593 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_testCloneSnapshot_clone,,1690229452224.dc99354a9c892a2852932acf3304fbbc.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1690229454593"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229454593"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229454593"}]},"ts":"1690229454593"} 2023-07-24 20:10:54,594 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE; CloseRegionProcedure dc99354a9c892a2852932acf3304fbbc, server=jenkins-hbase4.apache.org,34917,1690229443609}] 2023-07-24 20:10:54,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=95 2023-07-24 20:10:54,746 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close dc99354a9c892a2852932acf3304fbbc 2023-07-24 20:10:54,747 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing dc99354a9c892a2852932acf3304fbbc, disabling compactions & flushes 2023-07-24 20:10:54,747 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region Group_testCloneSnapshot_clone,,1690229452224.dc99354a9c892a2852932acf3304fbbc. 2023-07-24 20:10:54,747 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_testCloneSnapshot_clone,,1690229452224.dc99354a9c892a2852932acf3304fbbc. 2023-07-24 20:10:54,748 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on Group_testCloneSnapshot_clone,,1690229452224.dc99354a9c892a2852932acf3304fbbc. after waiting 0 ms 2023-07-24 20:10:54,748 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region Group_testCloneSnapshot_clone,,1690229452224.dc99354a9c892a2852932acf3304fbbc. 2023-07-24 20:10:54,752 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/default/Group_testCloneSnapshot_clone/dc99354a9c892a2852932acf3304fbbc/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-24 20:10:54,753 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed Group_testCloneSnapshot_clone,,1690229452224.dc99354a9c892a2852932acf3304fbbc. 2023-07-24 20:10:54,753 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for dc99354a9c892a2852932acf3304fbbc: 2023-07-24 20:10:54,754 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed dc99354a9c892a2852932acf3304fbbc 2023-07-24 20:10:54,755 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=96 updating hbase:meta row=dc99354a9c892a2852932acf3304fbbc, regionState=CLOSED 2023-07-24 20:10:54,755 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"Group_testCloneSnapshot_clone,,1690229452224.dc99354a9c892a2852932acf3304fbbc.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1690229454755"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229454755"}]},"ts":"1690229454755"} 2023-07-24 20:10:54,758 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-07-24 20:10:54,758 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; CloseRegionProcedure dc99354a9c892a2852932acf3304fbbc, server=jenkins-hbase4.apache.org,34917,1690229443609 in 163 msec 2023-07-24 20:10:54,760 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=96, resume processing ppid=95 2023-07-24 20:10:54,760 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, ppid=95, state=SUCCESS; TransitRegionStateProcedure table=Group_testCloneSnapshot_clone, region=dc99354a9c892a2852932acf3304fbbc, UNASSIGN in 168 msec 2023-07-24 20:10:54,761 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_testCloneSnapshot_clone","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690229454761"}]},"ts":"1690229454761"} 2023-07-24 20:10:54,763 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=Group_testCloneSnapshot_clone, state=DISABLED in hbase:meta 2023-07-24 20:10:54,765 INFO [PEWorker-2] procedure.DisableTableProcedure(305): Set Group_testCloneSnapshot_clone to state=DISABLED 2023-07-24 20:10:54,767 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=95, state=SUCCESS; DisableTableProcedure table=Group_testCloneSnapshot_clone in 184 msec 2023-07-24 20:10:54,888 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=95 2023-07-24 20:10:54,888 INFO [Listener at localhost/44145] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:Group_testCloneSnapshot_clone, procId: 95 completed 2023-07-24 20:10:54,889 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete Group_testCloneSnapshot_clone 2023-07-24 20:10:54,890 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] procedure2.ProcedureExecutor(1029): Stored pid=98, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=Group_testCloneSnapshot_clone 2023-07-24 20:10:54,893 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=98, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=Group_testCloneSnapshot_clone 2023-07-24 20:10:54,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint(577): Removing deleted table 'Group_testCloneSnapshot_clone' from rsgroup 'default' 2023-07-24 20:10:54,895 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=98, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=Group_testCloneSnapshot_clone 2023-07-24 20:10:54,898 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:54,899 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCloneSnapshot_clone/dc99354a9c892a2852932acf3304fbbc 2023-07-24 20:10:54,900 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:54,901 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-24 20:10:54,901 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCloneSnapshot_clone/dc99354a9c892a2852932acf3304fbbc/recovered.edits, FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCloneSnapshot_clone/dc99354a9c892a2852932acf3304fbbc/test] 2023-07-24 20:10:54,906 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=98 2023-07-24 20:10:54,907 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCloneSnapshot_clone/dc99354a9c892a2852932acf3304fbbc/recovered.edits/4.seqid to hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/archive/data/default/Group_testCloneSnapshot_clone/dc99354a9c892a2852932acf3304fbbc/recovered.edits/4.seqid 2023-07-24 20:10:54,909 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/default/Group_testCloneSnapshot_clone/dc99354a9c892a2852932acf3304fbbc 2023-07-24 20:10:54,909 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived Group_testCloneSnapshot_clone regions 2023-07-24 20:10:54,912 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=98, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=Group_testCloneSnapshot_clone 2023-07-24 20:10:54,914 WARN [PEWorker-4] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of Group_testCloneSnapshot_clone from hbase:meta 2023-07-24 20:10:54,918 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(421): Removing 'Group_testCloneSnapshot_clone' descriptor. 2023-07-24 20:10:54,920 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=98, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=Group_testCloneSnapshot_clone 2023-07-24 20:10:54,920 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(411): Removing 'Group_testCloneSnapshot_clone' from region states. 2023-07-24 20:10:54,920 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCloneSnapshot_clone,,1690229452224.dc99354a9c892a2852932acf3304fbbc.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690229454920"}]},"ts":"9223372036854775807"} 2023-07-24 20:10:54,922 INFO [PEWorker-4] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-07-24 20:10:54,922 DEBUG [PEWorker-4] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => dc99354a9c892a2852932acf3304fbbc, NAME => 'Group_testCloneSnapshot_clone,,1690229452224.dc99354a9c892a2852932acf3304fbbc.', STARTKEY => '', ENDKEY => ''}] 2023-07-24 20:10:54,922 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(415): Marking 'Group_testCloneSnapshot_clone' as deleted. 2023-07-24 20:10:54,922 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_testCloneSnapshot_clone","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1690229454922"}]},"ts":"9223372036854775807"} 2023-07-24 20:10:54,927 INFO [PEWorker-4] hbase.MetaTableAccessor(1658): Deleted table Group_testCloneSnapshot_clone state from META 2023-07-24 20:10:54,929 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(130): Finished pid=98, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=Group_testCloneSnapshot_clone 2023-07-24 20:10:54,930 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=98, state=SUCCESS; DeleteTableProcedure table=Group_testCloneSnapshot_clone in 40 msec 2023-07-24 20:10:55,007 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=98 2023-07-24 20:10:55,007 INFO [Listener at localhost/44145] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:Group_testCloneSnapshot_clone, procId: 98 completed 2023-07-24 20:10:55,011 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:55,011 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:55,012 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-24 20:10:55,013 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-24 20:10:55,013 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-24 20:10:55,014 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-24 20:10:55,014 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-24 20:10:55,015 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-24 20:10:55,019 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:55,019 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-24 20:10:55,021 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-24 20:10:55,025 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase(152): Restoring servers: 0 2023-07-24 20:10:55,026 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-24 20:10:55,028 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:55,029 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:55,031 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-24 20:10:55,034 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-24 20:10:55,039 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:55,039 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:55,044 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:39247] to rsgroup master 2023-07-24 20:10:55,044 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-24 20:10:55,044 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.CallRunner(144): callId: 569 service: MasterService methodName: ExecMasterService size: 119 connection: 172.31.14.131:36610 deadline: 1690230655043, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. 2023-07-24 20:10:55,044 WARN [Listener at localhost/44145] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.afterMethod(TestRSGroupsBasics.java:82) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-24 20:10:55,047 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:10:55,048 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:55,048 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:55,048 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:34483, jenkins-hbase4.apache.org:34917, jenkins-hbase4.apache.org:39791, jenkins-hbase4.apache.org:45209], Tables:[hbase:meta, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-24 20:10:55,049 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-24 20:10:55,050 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-24 20:10:55,078 INFO [Listener at localhost/44145] hbase.ResourceChecker(175): after: rsgroup.TestRSGroupsBasics#testCloneSnapshot Thread=513 (was 508) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,34917,1690229443609' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,45209,1690229445078' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/cluster_82e812c0-1679-84b5-414c-93cfc125cc3f/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4636fd6e-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,39791,1690229440084' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,34483,1690229439858' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xb35bd70-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1922791765_17 at /127.0.0.1:55928 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,39247,1690229437650)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/cluster_82e812c0-1679-84b5-414c-93cfc125cc3f/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1922791765_17 at /127.0.0.1:44430 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_743653681_17 at /127.0.0.1:46176 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=791 (was 792), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=408 (was 426), ProcessCount=176 (was 176), AvailableMemoryMB=5736 (was 5746) 2023-07-24 20:10:55,079 WARN [Listener at localhost/44145] hbase.ResourceChecker(130): Thread=513 is superior to 500 2023-07-24 20:10:55,100 INFO [Listener at localhost/44145] hbase.ResourceChecker(147): before: rsgroup.TestRSGroupsBasics#testCreateWhenRsgroupNoOnlineServers Thread=513, OpenFileDescriptor=791, MaxFileDescriptor=60000, SystemLoadAverage=408, ProcessCount=176, AvailableMemoryMB=5734 2023-07-24 20:10:55,100 WARN [Listener at localhost/44145] hbase.ResourceChecker(130): Thread=513 is superior to 500 2023-07-24 20:10:55,100 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase(132): testCreateWhenRsgroupNoOnlineServers 2023-07-24 20:10:55,106 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:55,106 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:55,108 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-24 20:10:55,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-24 20:10:55,108 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-24 20:10:55,109 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-24 20:10:55,109 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-24 20:10:55,110 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-24 20:10:55,115 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:55,115 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-24 20:10:55,117 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-24 20:10:55,121 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase(152): Restoring servers: 0 2023-07-24 20:10:55,121 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-24 20:10:55,124 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:55,124 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:55,125 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-24 20:10:55,127 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-24 20:10:55,132 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:55,132 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:55,135 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:39247] to rsgroup master 2023-07-24 20:10:55,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-24 20:10:55,136 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.CallRunner(144): callId: 597 service: MasterService methodName: ExecMasterService size: 119 connection: 172.31.14.131:36610 deadline: 1690230655135, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. 2023-07-24 20:10:55,136 WARN [Listener at localhost/44145] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.setUpBeforeMethod(TestRSGroupsBase.java:136) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.beforeMethod(TestRSGroupsBasics.java:77) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-24 20:10:55,138 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:10:55,138 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:55,138 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:55,139 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:34483, jenkins-hbase4.apache.org:34917, jenkins-hbase4.apache.org:39791, jenkins-hbase4.apache.org:45209], Tables:[hbase:meta, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-24 20:10:55,140 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-24 20:10:55,140 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-24 20:10:55,140 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBasics(141): testCreateWhenRsgroupNoOnlineServers 2023-07-24 20:10:55,141 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-24 20:10:55,141 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-24 20:10:55,142 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup appInfo 2023-07-24 20:10:55,147 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:55,148 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:55,149 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/appInfo 2023-07-24 20:10:55,153 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 6 2023-07-24 20:10:55,159 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-24 20:10:55,162 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:55,163 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:55,166 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:34483] to rsgroup appInfo 2023-07-24 20:10:55,168 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:55,169 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:55,173 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/appInfo 2023-07-24 20:10:55,174 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 6 2023-07-24 20:10:55,175 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(286): Moving 0 region(s) to group default, current retry=0 2023-07-24 20:10:55,175 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(261): All regions from [jenkins-hbase4.apache.org,34483,1690229439858] are moved back to default 2023-07-24 20:10:55,175 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(438): Move servers done: default => appInfo 2023-07-24 20:10:55,176 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-24 20:10:55,185 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:55,186 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:55,188 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=appInfo 2023-07-24 20:10:55,188 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-24 20:10:55,197 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/draining 2023-07-24 20:10:55,198 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.ServerManager(636): Server jenkins-hbase4.apache.org,34483,1690229439858 added to draining server list. 2023-07-24 20:10:55,198 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/draining/jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:55,200 WARN [zk-event-processor-pool-0] master.ServerManager(632): Server jenkins-hbase4.apache.org,34483,1690229439858 is already in the draining server list.Ignoring request to add it again. 2023-07-24 20:10:55,200 INFO [zk-event-processor-pool-0] master.DrainingServerTracker(92): Draining RS node created, adding to list [jenkins-hbase4.apache.org,34483,1690229439858] 2023-07-24 20:10:55,201 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.HMaster$15(3014): Client=jenkins//172.31.14.131 creating {NAME => 'Group_ns', hbase.rsgroup.name => 'appInfo'} 2023-07-24 20:10:55,202 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] procedure2.ProcedureExecutor(1029): Stored pid=99, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=Group_ns 2023-07-24 20:10:55,206 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=99 2023-07-24 20:10:55,210 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-07-24 20:10:55,213 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=99, state=SUCCESS; CreateNamespaceProcedure, namespace=Group_ns in 10 msec 2023-07-24 20:10:55,307 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=99 2023-07-24 20:10:55,309 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'Group_ns:testCreateWhenRsgroupNoOnlineServers', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-24 20:10:55,310 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] procedure2.ProcedureExecutor(1029): Stored pid=100, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-24 20:10:55,312 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=100, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers execute state=CREATE_TABLE_PRE_OPERATION 2023-07-24 20:10:55,313 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "Group_ns" qualifier: "testCreateWhenRsgroupNoOnlineServers" procId is: 100 2023-07-24 20:10:55,314 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=100 2023-07-24 20:10:55,328 INFO [PEWorker-3] procedure2.ProcedureExecutor(1528): Rolled back pid=100, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-create-table:org.apache.hadoop.hbase.HBaseIOException: No online servers in the rsgroup appInfo which table Group_ns:testCreateWhenRsgroupNoOnlineServers belongs to; CreateTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers exec-time=18 msec 2023-07-24 20:10:55,415 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=100 2023-07-24 20:10:55,417 INFO [Listener at localhost/44145] client.HBaseAdmin$TableFuture(3548): Operation: CREATE, Table Name: Group_ns:testCreateWhenRsgroupNoOnlineServers, procId: 100 failed with No online servers in the rsgroup appInfo which table Group_ns:testCreateWhenRsgroupNoOnlineServers belongs to 2023-07-24 20:10:55,418 DEBUG [Listener at localhost/44145] rsgroup.TestRSGroupsBasics(162): create table error org.apache.hadoop.hbase.HBaseIOException: No online servers in the rsgroup appInfo which table Group_ns:testCreateWhenRsgroupNoOnlineServers belongs to at java.lang.Thread.getStackTrace(Thread.java:1564) at org.apache.hadoop.hbase.util.FutureUtils.setStackTrace(FutureUtils.java:130) at org.apache.hadoop.hbase.util.FutureUtils.rethrow(FutureUtils.java:149) at org.apache.hadoop.hbase.util.FutureUtils.get(FutureUtils.java:186) at org.apache.hadoop.hbase.client.Admin.createTable(Admin.java:302) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.testCreateWhenRsgroupNoOnlineServers(TestRSGroupsBasics.java:159) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) at --------Future.get--------(Unknown Source) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint.validateRSGroup(RSGroupAdminEndpoint.java:540) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint.moveTableToValidRSGroup(RSGroupAdminEndpoint.java:529) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint.preCreateTableAction(RSGroupAdminEndpoint.java:501) at org.apache.hadoop.hbase.master.MasterCoprocessorHost$16.call(MasterCoprocessorHost.java:371) at org.apache.hadoop.hbase.master.MasterCoprocessorHost$16.call(MasterCoprocessorHost.java:368) at org.apache.hadoop.hbase.coprocessor.CoprocessorHost$ObserverOperationWithoutResult.callObserver(CoprocessorHost.java:558) at org.apache.hadoop.hbase.coprocessor.CoprocessorHost.execOperation(CoprocessorHost.java:631) at org.apache.hadoop.hbase.master.MasterCoprocessorHost.preCreateTableAction(MasterCoprocessorHost.java:368) at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.preCreate(CreateTableProcedure.java:267) at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:93) at org.apache.hadoop.hbase.master.procedure.CreateTableProcedure.executeFromState(CreateTableProcedure.java:53) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 20:10:55,424 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/draining/jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:55,424 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/draining 2023-07-24 20:10:55,425 INFO [zk-event-processor-pool-0] master.DrainingServerTracker(109): Draining RS node deleted, removing from list [jenkins-hbase4.apache.org,34483,1690229439858] 2023-07-24 20:10:55,428 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.HMaster$4(2112): Client=jenkins//172.31.14.131 create 'Group_ns:testCreateWhenRsgroupNoOnlineServers', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-24 20:10:55,429 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-24 20:10:55,431 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=101, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers execute state=CREATE_TABLE_PRE_OPERATION 2023-07-24 20:10:55,431 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(700): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "Group_ns" qualifier: "testCreateWhenRsgroupNoOnlineServers" procId is: 101 2023-07-24 20:10:55,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=101 2023-07-24 20:10:55,433 DEBUG [PEWorker-1] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:55,433 DEBUG [PEWorker-1] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:55,434 DEBUG [PEWorker-1] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/appInfo 2023-07-24 20:10:55,434 DEBUG [PEWorker-1] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 6 2023-07-24 20:10:55,437 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=101, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-24 20:10:55,439 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/Group_ns/testCreateWhenRsgroupNoOnlineServers/468761dcbd2ba29c6a3f5ddf8c5f09dc 2023-07-24 20:10:55,440 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/Group_ns/testCreateWhenRsgroupNoOnlineServers/468761dcbd2ba29c6a3f5ddf8c5f09dc empty. 2023-07-24 20:10:55,440 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/Group_ns/testCreateWhenRsgroupNoOnlineServers/468761dcbd2ba29c6a3f5ddf8c5f09dc 2023-07-24 20:10:55,440 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived Group_ns:testCreateWhenRsgroupNoOnlineServers regions 2023-07-24 20:10:55,458 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/Group_ns/testCreateWhenRsgroupNoOnlineServers/.tabledesc/.tableinfo.0000000001 2023-07-24 20:10:55,459 INFO [RegionOpenAndInit-Group_ns:testCreateWhenRsgroupNoOnlineServers-pool-0] regionserver.HRegion(7675): creating {ENCODED => 468761dcbd2ba29c6a3f5ddf8c5f09dc, NAME => 'Group_ns:testCreateWhenRsgroupNoOnlineServers,,1690229455428.468761dcbd2ba29c6a3f5ddf8c5f09dc.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='Group_ns:testCreateWhenRsgroupNoOnlineServers', {NAME => 'f', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp 2023-07-24 20:10:55,469 DEBUG [RegionOpenAndInit-Group_ns:testCreateWhenRsgroupNoOnlineServers-pool-0] regionserver.HRegion(866): Instantiated Group_ns:testCreateWhenRsgroupNoOnlineServers,,1690229455428.468761dcbd2ba29c6a3f5ddf8c5f09dc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:55,469 DEBUG [RegionOpenAndInit-Group_ns:testCreateWhenRsgroupNoOnlineServers-pool-0] regionserver.HRegion(1604): Closing 468761dcbd2ba29c6a3f5ddf8c5f09dc, disabling compactions & flushes 2023-07-24 20:10:55,469 INFO [RegionOpenAndInit-Group_ns:testCreateWhenRsgroupNoOnlineServers-pool-0] regionserver.HRegion(1626): Closing region Group_ns:testCreateWhenRsgroupNoOnlineServers,,1690229455428.468761dcbd2ba29c6a3f5ddf8c5f09dc. 2023-07-24 20:10:55,469 DEBUG [RegionOpenAndInit-Group_ns:testCreateWhenRsgroupNoOnlineServers-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_ns:testCreateWhenRsgroupNoOnlineServers,,1690229455428.468761dcbd2ba29c6a3f5ddf8c5f09dc. 2023-07-24 20:10:55,469 DEBUG [RegionOpenAndInit-Group_ns:testCreateWhenRsgroupNoOnlineServers-pool-0] regionserver.HRegion(1714): Acquired close lock on Group_ns:testCreateWhenRsgroupNoOnlineServers,,1690229455428.468761dcbd2ba29c6a3f5ddf8c5f09dc. after waiting 0 ms 2023-07-24 20:10:55,469 DEBUG [RegionOpenAndInit-Group_ns:testCreateWhenRsgroupNoOnlineServers-pool-0] regionserver.HRegion(1724): Updates disabled for region Group_ns:testCreateWhenRsgroupNoOnlineServers,,1690229455428.468761dcbd2ba29c6a3f5ddf8c5f09dc. 2023-07-24 20:10:55,469 INFO [RegionOpenAndInit-Group_ns:testCreateWhenRsgroupNoOnlineServers-pool-0] regionserver.HRegion(1838): Closed Group_ns:testCreateWhenRsgroupNoOnlineServers,,1690229455428.468761dcbd2ba29c6a3f5ddf8c5f09dc. 2023-07-24 20:10:55,469 DEBUG [RegionOpenAndInit-Group_ns:testCreateWhenRsgroupNoOnlineServers-pool-0] regionserver.HRegion(1558): Region close journal for 468761dcbd2ba29c6a3f5ddf8c5f09dc: 2023-07-24 20:10:55,471 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=101, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers execute state=CREATE_TABLE_ADD_TO_META 2023-07-24 20:10:55,472 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"Group_ns:testCreateWhenRsgroupNoOnlineServers,,1690229455428.468761dcbd2ba29c6a3f5ddf8c5f09dc.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229455472"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229455472"}]},"ts":"1690229455472"} 2023-07-24 20:10:55,474 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-24 20:10:55,474 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=101, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-24 20:10:55,474 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_ns:testCreateWhenRsgroupNoOnlineServers","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690229455474"}]},"ts":"1690229455474"} 2023-07-24 20:10:55,476 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=Group_ns:testCreateWhenRsgroupNoOnlineServers, state=ENABLING in hbase:meta 2023-07-24 20:10:55,479 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=102, ppid=101, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers, region=468761dcbd2ba29c6a3f5ddf8c5f09dc, ASSIGN}] 2023-07-24 20:10:55,480 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=102, ppid=101, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers, region=468761dcbd2ba29c6a3f5ddf8c5f09dc, ASSIGN 2023-07-24 20:10:55,481 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=102, ppid=101, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers, region=468761dcbd2ba29c6a3f5ddf8c5f09dc, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,34483,1690229439858; forceNewPlan=false, retain=false 2023-07-24 20:10:55,533 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=101 2023-07-24 20:10:55,632 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=102 updating hbase:meta row=468761dcbd2ba29c6a3f5ddf8c5f09dc, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:55,633 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_ns:testCreateWhenRsgroupNoOnlineServers,,1690229455428.468761dcbd2ba29c6a3f5ddf8c5f09dc.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229455632"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229455632"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229455632"}]},"ts":"1690229455632"} 2023-07-24 20:10:55,637 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE; OpenRegionProcedure 468761dcbd2ba29c6a3f5ddf8c5f09dc, server=jenkins-hbase4.apache.org,34483,1690229439858}] 2023-07-24 20:10:55,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=101 2023-07-24 20:10:55,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open Group_ns:testCreateWhenRsgroupNoOnlineServers,,1690229455428.468761dcbd2ba29c6a3f5ddf8c5f09dc. 2023-07-24 20:10:55,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 468761dcbd2ba29c6a3f5ddf8c5f09dc, NAME => 'Group_ns:testCreateWhenRsgroupNoOnlineServers,,1690229455428.468761dcbd2ba29c6a3f5ddf8c5f09dc.', STARTKEY => '', ENDKEY => ''} 2023-07-24 20:10:55,793 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCreateWhenRsgroupNoOnlineServers 468761dcbd2ba29c6a3f5ddf8c5f09dc 2023-07-24 20:10:55,793 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated Group_ns:testCreateWhenRsgroupNoOnlineServers,,1690229455428.468761dcbd2ba29c6a3f5ddf8c5f09dc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:10:55,793 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 468761dcbd2ba29c6a3f5ddf8c5f09dc 2023-07-24 20:10:55,793 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 468761dcbd2ba29c6a3f5ddf8c5f09dc 2023-07-24 20:10:55,794 INFO [StoreOpener-468761dcbd2ba29c6a3f5ddf8c5f09dc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f of region 468761dcbd2ba29c6a3f5ddf8c5f09dc 2023-07-24 20:10:55,796 DEBUG [StoreOpener-468761dcbd2ba29c6a3f5ddf8c5f09dc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/Group_ns/testCreateWhenRsgroupNoOnlineServers/468761dcbd2ba29c6a3f5ddf8c5f09dc/f 2023-07-24 20:10:55,796 DEBUG [StoreOpener-468761dcbd2ba29c6a3f5ddf8c5f09dc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/Group_ns/testCreateWhenRsgroupNoOnlineServers/468761dcbd2ba29c6a3f5ddf8c5f09dc/f 2023-07-24 20:10:55,796 INFO [StoreOpener-468761dcbd2ba29c6a3f5ddf8c5f09dc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 468761dcbd2ba29c6a3f5ddf8c5f09dc columnFamilyName f 2023-07-24 20:10:55,797 INFO [StoreOpener-468761dcbd2ba29c6a3f5ddf8c5f09dc-1] regionserver.HStore(310): Store=468761dcbd2ba29c6a3f5ddf8c5f09dc/f, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:10:55,797 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/Group_ns/testCreateWhenRsgroupNoOnlineServers/468761dcbd2ba29c6a3f5ddf8c5f09dc 2023-07-24 20:10:55,798 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/Group_ns/testCreateWhenRsgroupNoOnlineServers/468761dcbd2ba29c6a3f5ddf8c5f09dc 2023-07-24 20:10:55,800 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 468761dcbd2ba29c6a3f5ddf8c5f09dc 2023-07-24 20:10:55,802 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/Group_ns/testCreateWhenRsgroupNoOnlineServers/468761dcbd2ba29c6a3f5ddf8c5f09dc/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 20:10:55,803 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 468761dcbd2ba29c6a3f5ddf8c5f09dc; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=11289774240, jitterRate=0.051442161202430725}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 20:10:55,803 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 468761dcbd2ba29c6a3f5ddf8c5f09dc: 2023-07-24 20:10:55,803 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for Group_ns:testCreateWhenRsgroupNoOnlineServers,,1690229455428.468761dcbd2ba29c6a3f5ddf8c5f09dc., pid=103, masterSystemTime=1690229455788 2023-07-24 20:10:55,805 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for Group_ns:testCreateWhenRsgroupNoOnlineServers,,1690229455428.468761dcbd2ba29c6a3f5ddf8c5f09dc. 2023-07-24 20:10:55,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened Group_ns:testCreateWhenRsgroupNoOnlineServers,,1690229455428.468761dcbd2ba29c6a3f5ddf8c5f09dc. 2023-07-24 20:10:55,805 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=102 updating hbase:meta row=468761dcbd2ba29c6a3f5ddf8c5f09dc, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:55,805 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"Group_ns:testCreateWhenRsgroupNoOnlineServers,,1690229455428.468761dcbd2ba29c6a3f5ddf8c5f09dc.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229455805"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690229455805"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690229455805"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690229455805"}]},"ts":"1690229455805"} 2023-07-24 20:10:55,808 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-07-24 20:10:55,808 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; OpenRegionProcedure 468761dcbd2ba29c6a3f5ddf8c5f09dc, server=jenkins-hbase4.apache.org,34483,1690229439858 in 173 msec 2023-07-24 20:10:55,810 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=102, resume processing ppid=101 2023-07-24 20:10:55,810 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, ppid=101, state=SUCCESS; TransitRegionStateProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers, region=468761dcbd2ba29c6a3f5ddf8c5f09dc, ASSIGN in 330 msec 2023-07-24 20:10:55,810 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=101, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-24 20:10:55,811 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_ns:testCreateWhenRsgroupNoOnlineServers","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690229455810"}]},"ts":"1690229455810"} 2023-07-24 20:10:55,812 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=Group_ns:testCreateWhenRsgroupNoOnlineServers, state=ENABLED in hbase:meta 2023-07-24 20:10:55,814 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=101, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers execute state=CREATE_TABLE_POST_OPERATION 2023-07-24 20:10:55,815 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; CreateTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers in 386 msec 2023-07-24 20:10:56,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=101 2023-07-24 20:10:56,035 INFO [Listener at localhost/44145] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: Group_ns:testCreateWhenRsgroupNoOnlineServers, procId: 101 completed 2023-07-24 20:10:56,036 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:10:56,041 INFO [Listener at localhost/44145] client.HBaseAdmin$15(890): Started disable of Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-24 20:10:56,041 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-24 20:10:56,042 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] procedure2.ProcedureExecutor(1029): Stored pid=104, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-24 20:10:56,045 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=104 2023-07-24 20:10:56,047 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_ns:testCreateWhenRsgroupNoOnlineServers","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690229456047"}]},"ts":"1690229456047"} 2023-07-24 20:10:56,048 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=Group_ns:testCreateWhenRsgroupNoOnlineServers, state=DISABLING in hbase:meta 2023-07-24 20:10:56,050 INFO [PEWorker-4] procedure.DisableTableProcedure(293): Set Group_ns:testCreateWhenRsgroupNoOnlineServers to state=DISABLING 2023-07-24 20:10:56,055 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=105, ppid=104, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers, region=468761dcbd2ba29c6a3f5ddf8c5f09dc, UNASSIGN}] 2023-07-24 20:10:56,057 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=105, ppid=104, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers, region=468761dcbd2ba29c6a3f5ddf8c5f09dc, UNASSIGN 2023-07-24 20:10:56,059 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=105 updating hbase:meta row=468761dcbd2ba29c6a3f5ddf8c5f09dc, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:56,059 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"Group_ns:testCreateWhenRsgroupNoOnlineServers,,1690229455428.468761dcbd2ba29c6a3f5ddf8c5f09dc.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229456058"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229456058"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229456058"}]},"ts":"1690229456058"} 2023-07-24 20:10:56,062 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE; CloseRegionProcedure 468761dcbd2ba29c6a3f5ddf8c5f09dc, server=jenkins-hbase4.apache.org,34483,1690229439858}] 2023-07-24 20:10:56,147 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=104 2023-07-24 20:10:56,216 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 468761dcbd2ba29c6a3f5ddf8c5f09dc 2023-07-24 20:10:56,217 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 468761dcbd2ba29c6a3f5ddf8c5f09dc, disabling compactions & flushes 2023-07-24 20:10:56,218 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region Group_ns:testCreateWhenRsgroupNoOnlineServers,,1690229455428.468761dcbd2ba29c6a3f5ddf8c5f09dc. 2023-07-24 20:10:56,218 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on Group_ns:testCreateWhenRsgroupNoOnlineServers,,1690229455428.468761dcbd2ba29c6a3f5ddf8c5f09dc. 2023-07-24 20:10:56,218 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on Group_ns:testCreateWhenRsgroupNoOnlineServers,,1690229455428.468761dcbd2ba29c6a3f5ddf8c5f09dc. after waiting 0 ms 2023-07-24 20:10:56,218 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region Group_ns:testCreateWhenRsgroupNoOnlineServers,,1690229455428.468761dcbd2ba29c6a3f5ddf8c5f09dc. 2023-07-24 20:10:56,222 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/Group_ns/testCreateWhenRsgroupNoOnlineServers/468761dcbd2ba29c6a3f5ddf8c5f09dc/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-24 20:10:56,223 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed Group_ns:testCreateWhenRsgroupNoOnlineServers,,1690229455428.468761dcbd2ba29c6a3f5ddf8c5f09dc. 2023-07-24 20:10:56,223 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 468761dcbd2ba29c6a3f5ddf8c5f09dc: 2023-07-24 20:10:56,225 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 468761dcbd2ba29c6a3f5ddf8c5f09dc 2023-07-24 20:10:56,226 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=105 updating hbase:meta row=468761dcbd2ba29c6a3f5ddf8c5f09dc, regionState=CLOSED 2023-07-24 20:10:56,226 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"Group_ns:testCreateWhenRsgroupNoOnlineServers,,1690229455428.468761dcbd2ba29c6a3f5ddf8c5f09dc.","families":{"info":[{"qualifier":"regioninfo","vlen":71,"tag":[],"timestamp":"1690229456226"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229456226"}]},"ts":"1690229456226"} 2023-07-24 20:10:56,230 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-07-24 20:10:56,230 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; CloseRegionProcedure 468761dcbd2ba29c6a3f5ddf8c5f09dc, server=jenkins-hbase4.apache.org,34483,1690229439858 in 166 msec 2023-07-24 20:10:56,232 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=105, resume processing ppid=104 2023-07-24 20:10:56,232 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=105, ppid=104, state=SUCCESS; TransitRegionStateProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers, region=468761dcbd2ba29c6a3f5ddf8c5f09dc, UNASSIGN in 175 msec 2023-07-24 20:10:56,233 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"Group_ns:testCreateWhenRsgroupNoOnlineServers","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690229456233"}]},"ts":"1690229456233"} 2023-07-24 20:10:56,234 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=Group_ns:testCreateWhenRsgroupNoOnlineServers, state=DISABLED in hbase:meta 2023-07-24 20:10:56,237 INFO [PEWorker-4] procedure.DisableTableProcedure(305): Set Group_ns:testCreateWhenRsgroupNoOnlineServers to state=DISABLED 2023-07-24 20:10:56,240 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=104, state=SUCCESS; DisableTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers in 196 msec 2023-07-24 20:10:56,348 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=104 2023-07-24 20:10:56,348 INFO [Listener at localhost/44145] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: Group_ns:testCreateWhenRsgroupNoOnlineServers, procId: 104 completed 2023-07-24 20:10:56,349 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-24 20:10:56,350 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] procedure2.ProcedureExecutor(1029): Stored pid=107, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-24 20:10:56,352 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=107, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-24 20:10:56,352 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint(577): Removing deleted table 'Group_ns:testCreateWhenRsgroupNoOnlineServers' from rsgroup 'appInfo' 2023-07-24 20:10:56,353 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=107, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-24 20:10:56,354 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:56,355 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:56,355 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/appInfo 2023-07-24 20:10:56,356 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 6 2023-07-24 20:10:56,357 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/Group_ns/testCreateWhenRsgroupNoOnlineServers/468761dcbd2ba29c6a3f5ddf8c5f09dc 2023-07-24 20:10:56,358 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=107 2023-07-24 20:10:56,359 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/Group_ns/testCreateWhenRsgroupNoOnlineServers/468761dcbd2ba29c6a3f5ddf8c5f09dc/f, FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/Group_ns/testCreateWhenRsgroupNoOnlineServers/468761dcbd2ba29c6a3f5ddf8c5f09dc/recovered.edits] 2023-07-24 20:10:56,365 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/Group_ns/testCreateWhenRsgroupNoOnlineServers/468761dcbd2ba29c6a3f5ddf8c5f09dc/recovered.edits/4.seqid to hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/archive/data/Group_ns/testCreateWhenRsgroupNoOnlineServers/468761dcbd2ba29c6a3f5ddf8c5f09dc/recovered.edits/4.seqid 2023-07-24 20:10:56,365 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/Group_ns/testCreateWhenRsgroupNoOnlineServers/468761dcbd2ba29c6a3f5ddf8c5f09dc 2023-07-24 20:10:56,365 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived Group_ns:testCreateWhenRsgroupNoOnlineServers regions 2023-07-24 20:10:56,368 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=107, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-24 20:10:56,370 WARN [PEWorker-5] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of Group_ns:testCreateWhenRsgroupNoOnlineServers from hbase:meta 2023-07-24 20:10:56,372 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(421): Removing 'Group_ns:testCreateWhenRsgroupNoOnlineServers' descriptor. 2023-07-24 20:10:56,373 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=107, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-24 20:10:56,373 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(411): Removing 'Group_ns:testCreateWhenRsgroupNoOnlineServers' from region states. 2023-07-24 20:10:56,373 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_ns:testCreateWhenRsgroupNoOnlineServers,,1690229455428.468761dcbd2ba29c6a3f5ddf8c5f09dc.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690229456373"}]},"ts":"9223372036854775807"} 2023-07-24 20:10:56,375 INFO [PEWorker-5] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-07-24 20:10:56,375 DEBUG [PEWorker-5] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 468761dcbd2ba29c6a3f5ddf8c5f09dc, NAME => 'Group_ns:testCreateWhenRsgroupNoOnlineServers,,1690229455428.468761dcbd2ba29c6a3f5ddf8c5f09dc.', STARTKEY => '', ENDKEY => ''}] 2023-07-24 20:10:56,375 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(415): Marking 'Group_ns:testCreateWhenRsgroupNoOnlineServers' as deleted. 2023-07-24 20:10:56,375 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"Group_ns:testCreateWhenRsgroupNoOnlineServers","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1690229456375"}]},"ts":"9223372036854775807"} 2023-07-24 20:10:56,376 INFO [PEWorker-5] hbase.MetaTableAccessor(1658): Deleted table Group_ns:testCreateWhenRsgroupNoOnlineServers state from META 2023-07-24 20:10:56,378 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(130): Finished pid=107, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-24 20:10:56,379 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=107, state=SUCCESS; DeleteTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers in 29 msec 2023-07-24 20:10:56,459 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=107 2023-07-24 20:10:56,460 INFO [Listener at localhost/44145] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: Group_ns:testCreateWhenRsgroupNoOnlineServers, procId: 107 completed 2023-07-24 20:10:56,463 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.HMaster$17(3086): Client=jenkins//172.31.14.131 delete Group_ns 2023-07-24 20:10:56,464 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_NAMESPACE_PREPARE; DeleteNamespaceProcedure, namespace=Group_ns 2023-07-24 20:10:56,466 INFO [PEWorker-3] procedure.DeleteNamespaceProcedure(73): pid=108, state=RUNNABLE:DELETE_NAMESPACE_PREPARE, locked=true; DeleteNamespaceProcedure, namespace=Group_ns 2023-07-24 20:10:56,469 INFO [PEWorker-3] procedure.DeleteNamespaceProcedure(73): pid=108, state=RUNNABLE:DELETE_NAMESPACE_DELETE_FROM_NS_TABLE, locked=true; DeleteNamespaceProcedure, namespace=Group_ns 2023-07-24 20:10:56,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=108 2023-07-24 20:10:56,471 INFO [PEWorker-3] procedure.DeleteNamespaceProcedure(73): pid=108, state=RUNNABLE:DELETE_NAMESPACE_REMOVE_FROM_ZK, locked=true; DeleteNamespaceProcedure, namespace=Group_ns 2023-07-24 20:10:56,472 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/namespace/Group_ns 2023-07-24 20:10:56,472 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-07-24 20:10:56,474 INFO [PEWorker-3] procedure.DeleteNamespaceProcedure(73): pid=108, state=RUNNABLE:DELETE_NAMESPACE_DELETE_DIRECTORIES, locked=true; DeleteNamespaceProcedure, namespace=Group_ns 2023-07-24 20:10:56,482 INFO [PEWorker-3] procedure.DeleteNamespaceProcedure(73): pid=108, state=RUNNABLE:DELETE_NAMESPACE_REMOVE_NAMESPACE_QUOTA, locked=true; DeleteNamespaceProcedure, namespace=Group_ns 2023-07-24 20:10:56,486 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteNamespaceProcedure, namespace=Group_ns in 18 msec 2023-07-24 20:10:56,570 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(1230): Checking to see if procedure is done pid=108 2023-07-24 20:10:56,571 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:56,571 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:56,572 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-24 20:10:56,572 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-24 20:10:56,572 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-24 20:10:56,573 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-24 20:10:56,573 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-24 20:10:56,574 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-24 20:10:56,578 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:56,579 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/appInfo 2023-07-24 20:10:56,579 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 5 2023-07-24 20:10:56,583 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-24 20:10:56,584 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-24 20:10:56,584 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-24 20:10:56,584 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-24 20:10:56,585 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:34483] to rsgroup default 2023-07-24 20:10:56,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:56,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/appInfo 2023-07-24 20:10:56,588 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-24 20:10:56,590 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(286): Moving 0 region(s) to group appInfo, current retry=0 2023-07-24 20:10:56,590 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(261): All regions from [jenkins-hbase4.apache.org,34483,1690229439858] are moved back to appInfo 2023-07-24 20:10:56,590 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(438): Move servers done: appInfo => default 2023-07-24 20:10:56,590 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-24 20:10:56,591 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup appInfo 2023-07-24 20:10:56,595 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:56,596 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-24 20:10:56,602 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-24 20:10:56,606 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase(152): Restoring servers: 0 2023-07-24 20:10:56,607 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-24 20:10:56,609 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:56,609 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:56,611 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-24 20:10:56,612 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-24 20:10:56,616 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:56,616 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:56,625 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:39247] to rsgroup master 2023-07-24 20:10:56,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-24 20:10:56,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.CallRunner(144): callId: 699 service: MasterService methodName: ExecMasterService size: 119 connection: 172.31.14.131:36610 deadline: 1690230656625, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. 2023-07-24 20:10:56,626 WARN [Listener at localhost/44145] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.afterMethod(TestRSGroupsBasics.java:82) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-24 20:10:56,628 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:10:56,629 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:56,629 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:56,630 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:34483, jenkins-hbase4.apache.org:34917, jenkins-hbase4.apache.org:39791, jenkins-hbase4.apache.org:45209], Tables:[hbase:meta, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-24 20:10:56,631 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-24 20:10:56,631 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-24 20:10:56,654 INFO [Listener at localhost/44145] hbase.ResourceChecker(175): after: rsgroup.TestRSGroupsBasics#testCreateWhenRsgroupNoOnlineServers Thread=514 (was 513) Potentially hanging thread: hconnection-0xb35bd70-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xb35bd70-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xb35bd70-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0xb35bd70-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1680613140_17 at /127.0.0.1:44430 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=791 (was 791), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=408 (was 408), ProcessCount=176 (was 176), AvailableMemoryMB=5715 (was 5734) 2023-07-24 20:10:56,654 WARN [Listener at localhost/44145] hbase.ResourceChecker(130): Thread=514 is superior to 500 2023-07-24 20:10:56,675 INFO [Listener at localhost/44145] hbase.ResourceChecker(147): before: rsgroup.TestRSGroupsBasics#testBasicStartUp Thread=514, OpenFileDescriptor=791, MaxFileDescriptor=60000, SystemLoadAverage=408, ProcessCount=176, AvailableMemoryMB=5713 2023-07-24 20:10:56,675 WARN [Listener at localhost/44145] hbase.ResourceChecker(130): Thread=514 is superior to 500 2023-07-24 20:10:56,676 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase(132): testBasicStartUp 2023-07-24 20:10:56,680 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:56,680 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:56,681 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-24 20:10:56,681 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-24 20:10:56,681 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-24 20:10:56,682 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-24 20:10:56,682 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-24 20:10:56,683 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-24 20:10:56,686 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:56,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-24 20:10:56,688 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-24 20:10:56,691 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase(152): Restoring servers: 0 2023-07-24 20:10:56,691 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-24 20:10:56,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:56,694 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:56,697 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-24 20:10:56,703 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-24 20:10:56,706 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:56,706 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:56,709 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:39247] to rsgroup master 2023-07-24 20:10:56,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-24 20:10:56,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.CallRunner(144): callId: 727 service: MasterService methodName: ExecMasterService size: 119 connection: 172.31.14.131:36610 deadline: 1690230656709, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. 2023-07-24 20:10:56,710 WARN [Listener at localhost/44145] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.setUpBeforeMethod(TestRSGroupsBase.java:136) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.beforeMethod(TestRSGroupsBasics.java:77) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-24 20:10:56,712 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:10:56,712 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:56,713 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:56,713 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:34483, jenkins-hbase4.apache.org:34917, jenkins-hbase4.apache.org:39791, jenkins-hbase4.apache.org:45209], Tables:[hbase:meta, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-24 20:10:56,714 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-24 20:10:56,714 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-24 20:10:56,715 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-24 20:10:56,715 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-24 20:10:56,719 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:56,719 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:56,720 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-24 20:10:56,720 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-24 20:10:56,720 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-24 20:10:56,721 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-24 20:10:56,721 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-24 20:10:56,723 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-24 20:10:56,730 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:56,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-24 20:10:56,733 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-24 20:10:56,736 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase(152): Restoring servers: 0 2023-07-24 20:10:56,737 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-24 20:10:56,739 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:56,739 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:56,741 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-24 20:10:56,747 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-24 20:10:56,750 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:56,750 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:56,752 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:39247] to rsgroup master 2023-07-24 20:10:56,752 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-24 20:10:56,753 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.CallRunner(144): callId: 757 service: MasterService methodName: ExecMasterService size: 119 connection: 172.31.14.131:36610 deadline: 1690230656752, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. 2023-07-24 20:10:56,753 WARN [Listener at localhost/44145] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.afterMethod(TestRSGroupsBasics.java:82) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-24 20:10:56,755 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:10:56,756 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:56,757 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:56,757 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:34483, jenkins-hbase4.apache.org:34917, jenkins-hbase4.apache.org:39791, jenkins-hbase4.apache.org:45209], Tables:[hbase:meta, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-24 20:10:56,759 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-24 20:10:56,760 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-24 20:10:56,785 INFO [Listener at localhost/44145] hbase.ResourceChecker(175): after: rsgroup.TestRSGroupsBasics#testBasicStartUp Thread=515 (was 514) Potentially hanging thread: hconnection-0xb35bd70-shared-pool-18 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=791 (was 791), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=408 (was 408), ProcessCount=176 (was 176), AvailableMemoryMB=5715 (was 5713) - AvailableMemoryMB LEAK? - 2023-07-24 20:10:56,785 WARN [Listener at localhost/44145] hbase.ResourceChecker(130): Thread=515 is superior to 500 2023-07-24 20:10:56,813 INFO [Listener at localhost/44145] hbase.ResourceChecker(147): before: rsgroup.TestRSGroupsBasics#testRSGroupsWithHBaseQuota Thread=515, OpenFileDescriptor=791, MaxFileDescriptor=60000, SystemLoadAverage=408, ProcessCount=176, AvailableMemoryMB=5716 2023-07-24 20:10:56,814 WARN [Listener at localhost/44145] hbase.ResourceChecker(130): Thread=515 is superior to 500 2023-07-24 20:10:56,814 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase(132): testRSGroupsWithHBaseQuota 2023-07-24 20:10:56,820 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:56,821 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:56,822 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-24 20:10:56,822 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-24 20:10:56,822 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-24 20:10:56,823 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-24 20:10:56,823 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-24 20:10:56,824 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-24 20:10:56,828 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:56,829 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-24 20:10:56,834 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-24 20:10:56,837 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase(152): Restoring servers: 0 2023-07-24 20:10:56,838 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-24 20:10:56,841 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:10:56,841 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:10:56,843 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-24 20:10:56,851 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-24 20:10:56,856 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:56,856 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:56,860 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:39247] to rsgroup master 2023-07-24 20:10:56,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-24 20:10:56,860 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] ipc.CallRunner(144): callId: 785 service: MasterService methodName: ExecMasterService size: 119 connection: 172.31.14.131:36610 deadline: 1690230656860, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. 2023-07-24 20:10:56,861 WARN [Listener at localhost/44145] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.GeneratedConstructorAccessor62.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.setUpBeforeMethod(TestRSGroupsBase.java:136) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.beforeMethod(TestRSGroupsBasics.java:77) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:39247 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-24 20:10:56,863 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:10:56,864 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:10:56,864 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:10:56,864 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:34483, jenkins-hbase4.apache.org:34917, jenkins-hbase4.apache.org:39791, jenkins-hbase4.apache.org:45209], Tables:[hbase:meta, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-24 20:10:56,865 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-24 20:10:56,865 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39247] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-24 20:10:56,866 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBasics(309): Shutting down cluster 2023-07-24 20:10:56,866 INFO [Listener at localhost/44145] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-07-24 20:10:56,866 DEBUG [Listener at localhost/44145] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6ecbabf0 to 127.0.0.1:62881 2023-07-24 20:10:56,866 DEBUG [Listener at localhost/44145] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:10:56,867 DEBUG [Listener at localhost/44145] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-24 20:10:56,867 DEBUG [Listener at localhost/44145] util.JVMClusterUtil(257): Found active master hash=1614239372, stopped=false 2023-07-24 20:10:56,867 DEBUG [Listener at localhost/44145] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint 2023-07-24 20:10:56,867 DEBUG [Listener at localhost/44145] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase$CPMasterObserver 2023-07-24 20:10:56,867 INFO [Listener at localhost/44145] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,39247,1690229437650 2023-07-24 20:10:56,869 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-24 20:10:56,869 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34917-0x10198f4f3ce000b, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-24 20:10:56,869 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-24 20:10:56,869 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:45209-0x10198f4f3ce000d, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-24 20:10:56,869 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-24 20:10:56,869 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 20:10:56,869 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 20:10:56,869 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45209-0x10198f4f3ce000d, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 20:10:56,869 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 20:10:56,869 INFO [Listener at localhost/44145] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 20:10:56,870 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34917-0x10198f4f3ce000b, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 20:10:56,870 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 20:10:56,870 DEBUG [Listener at localhost/44145] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4c458dfb to 127.0.0.1:62881 2023-07-24 20:10:56,871 DEBUG [Listener at localhost/44145] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:10:56,872 INFO [Listener at localhost/44145] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,34483,1690229439858' ***** 2023-07-24 20:10:56,872 INFO [Listener at localhost/44145] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-24 20:10:56,872 INFO [RS:4;jenkins-hbase4:45209] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,45209,1690229445078' ***** 2023-07-24 20:10:56,872 INFO [RS:4;jenkins-hbase4:45209] regionserver.HRegionServer(2311): STOPPED: Exiting; cluster shutdown set and not carrying any regions 2023-07-24 20:10:56,872 INFO [RS:0;jenkins-hbase4:34483] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,34483,1690229439858' ***** 2023-07-24 20:10:56,872 INFO [RS:0;jenkins-hbase4:34483] regionserver.HRegionServer(2311): STOPPED: Exiting; cluster shutdown set and not carrying any regions 2023-07-24 20:10:56,872 INFO [Listener at localhost/44145] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,39791,1690229440084' ***** 2023-07-24 20:10:56,873 INFO [Listener at localhost/44145] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-24 20:10:56,873 INFO [Listener at localhost/44145] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,34917,1690229443609' ***** 2023-07-24 20:10:56,873 INFO [Listener at localhost/44145] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-24 20:10:56,873 INFO [RS:2;jenkins-hbase4:39791] regionserver.HRegionServer(1109): Stopping infoServer 2023-07-24 20:10:56,873 INFO [RS:3;jenkins-hbase4:34917] regionserver.HRegionServer(1109): Stopping infoServer 2023-07-24 20:10:56,875 INFO [RS:4;jenkins-hbase4:45209] regionserver.HRegionServer(1109): Stopping infoServer 2023-07-24 20:10:56,875 INFO [RS:0;jenkins-hbase4:34483] regionserver.HRegionServer(1109): Stopping infoServer 2023-07-24 20:10:56,885 INFO [RS:4;jenkins-hbase4:45209] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.w.WebAppContext@25fb09ce{regionserver,/,null,STOPPED}{file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver} 2023-07-24 20:10:56,885 INFO [RS:2;jenkins-hbase4:39791] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.w.WebAppContext@1d722a93{regionserver,/,null,STOPPED}{file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver} 2023-07-24 20:10:56,885 INFO [RS:4;jenkins-hbase4:45209] server.AbstractConnector(383): Stopped ServerConnector@7790aa8{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-24 20:10:56,885 INFO [RS:4;jenkins-hbase4:45209] session.HouseKeeper(149): node0 Stopped scavenging 2023-07-24 20:10:56,886 INFO [RS:2;jenkins-hbase4:39791] server.AbstractConnector(383): Stopped ServerConnector@7df64ca9{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-24 20:10:56,886 INFO [RS:2;jenkins-hbase4:39791] session.HouseKeeper(149): node0 Stopped scavenging 2023-07-24 20:10:56,888 INFO [RS:2;jenkins-hbase4:39791] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@3062dffd{static,/static,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/static/,STOPPED} 2023-07-24 20:10:56,888 INFO [RS:4;jenkins-hbase4:45209] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@3bcb01cf{static,/static,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/static/,STOPPED} 2023-07-24 20:10:56,889 INFO [RS:3;jenkins-hbase4:34917] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.w.WebAppContext@2f83be5f{regionserver,/,null,STOPPED}{file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver} 2023-07-24 20:10:56,889 INFO [RS:0;jenkins-hbase4:34483] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.w.WebAppContext@50a284fe{regionserver,/,null,STOPPED}{file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver} 2023-07-24 20:10:56,890 INFO [RS:4;jenkins-hbase4:45209] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@64470437{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/hadoop.log.dir/,STOPPED} 2023-07-24 20:10:56,890 INFO [RS:0;jenkins-hbase4:34483] server.AbstractConnector(383): Stopped ServerConnector@5417460a{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-24 20:10:56,890 INFO [RS:2;jenkins-hbase4:39791] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@57741f78{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/hadoop.log.dir/,STOPPED} 2023-07-24 20:10:56,890 INFO [RS:3;jenkins-hbase4:34917] server.AbstractConnector(383): Stopped ServerConnector@41896d02{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-24 20:10:56,891 INFO [RS:3;jenkins-hbase4:34917] session.HouseKeeper(149): node0 Stopped scavenging 2023-07-24 20:10:56,890 INFO [RS:0;jenkins-hbase4:34483] session.HouseKeeper(149): node0 Stopped scavenging 2023-07-24 20:10:56,893 INFO [RS:2;jenkins-hbase4:39791] regionserver.HeapMemoryManager(220): Stopping 2023-07-24 20:10:56,894 INFO [RS:2;jenkins-hbase4:39791] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-24 20:10:56,894 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-24 20:10:56,894 INFO [RS:2;jenkins-hbase4:39791] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-24 20:10:56,894 INFO [RS:2;jenkins-hbase4:39791] regionserver.HRegionServer(3305): Received CLOSE for 324410cf2a29c2dca1a19e8570799d47 2023-07-24 20:10:56,894 INFO [RS:2;jenkins-hbase4:39791] regionserver.HRegionServer(3305): Received CLOSE for ec4f892f95beff2c1aa7863da43a18fa 2023-07-24 20:10:56,894 INFO [RS:2;jenkins-hbase4:39791] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:56,895 DEBUG [RS:2;jenkins-hbase4:39791] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x683ab3ae to 127.0.0.1:62881 2023-07-24 20:10:56,895 DEBUG [RS:2;jenkins-hbase4:39791] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:10:56,895 INFO [RS:2;jenkins-hbase4:39791] regionserver.HRegionServer(1474): Waiting on 2 regions to close 2023-07-24 20:10:56,896 DEBUG [RS:2;jenkins-hbase4:39791] regionserver.HRegionServer(1478): Online Regions={324410cf2a29c2dca1a19e8570799d47=hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47., ec4f892f95beff2c1aa7863da43a18fa=hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa.} 2023-07-24 20:10:56,895 INFO [RS:3;jenkins-hbase4:34917] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@8b44a53{static,/static,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/static/,STOPPED} 2023-07-24 20:10:56,895 INFO [RS:4;jenkins-hbase4:45209] regionserver.HeapMemoryManager(220): Stopping 2023-07-24 20:10:56,897 DEBUG [RS:2;jenkins-hbase4:39791] regionserver.HRegionServer(1504): Waiting on 324410cf2a29c2dca1a19e8570799d47, ec4f892f95beff2c1aa7863da43a18fa 2023-07-24 20:10:56,897 INFO [RS:4;jenkins-hbase4:45209] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-24 20:10:56,895 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 324410cf2a29c2dca1a19e8570799d47, disabling compactions & flushes 2023-07-24 20:10:56,895 INFO [RS:0;jenkins-hbase4:34483] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@5c57f5de{static,/static,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/static/,STOPPED} 2023-07-24 20:10:56,897 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. 2023-07-24 20:10:56,898 INFO [RS:0;jenkins-hbase4:34483] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@15af0305{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/hadoop.log.dir/,STOPPED} 2023-07-24 20:10:56,897 INFO [RS:4;jenkins-hbase4:45209] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-24 20:10:56,898 INFO [RS:4;jenkins-hbase4:45209] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:56,897 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-24 20:10:56,897 INFO [RS:3;jenkins-hbase4:34917] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@42f42ab7{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/hadoop.log.dir/,STOPPED} 2023-07-24 20:10:56,898 DEBUG [RS:4;jenkins-hbase4:45209] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x44764448 to 127.0.0.1:62881 2023-07-24 20:10:56,899 DEBUG [RS:4;jenkins-hbase4:45209] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:10:56,899 INFO [RS:4;jenkins-hbase4:45209] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,45209,1690229445078; all regions closed. 2023-07-24 20:10:56,898 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. 2023-07-24 20:10:56,899 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. after waiting 0 ms 2023-07-24 20:10:56,899 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. 2023-07-24 20:10:56,900 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 324410cf2a29c2dca1a19e8570799d47 1/1 column families, dataSize=365 B heapSize=1.13 KB 2023-07-24 20:10:56,900 INFO [RS:0;jenkins-hbase4:34483] regionserver.HeapMemoryManager(220): Stopping 2023-07-24 20:10:56,900 INFO [RS:0;jenkins-hbase4:34483] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-24 20:10:56,900 INFO [RS:0;jenkins-hbase4:34483] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-24 20:10:56,900 INFO [RS:0;jenkins-hbase4:34483] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:56,900 DEBUG [RS:0;jenkins-hbase4:34483] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x507a6840 to 127.0.0.1:62881 2023-07-24 20:10:56,901 DEBUG [RS:0;jenkins-hbase4:34483] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:10:56,901 INFO [RS:0;jenkins-hbase4:34483] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,34483,1690229439858; all regions closed. 2023-07-24 20:10:56,900 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-24 20:10:56,906 INFO [RS:3;jenkins-hbase4:34917] regionserver.HeapMemoryManager(220): Stopping 2023-07-24 20:10:56,907 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-24 20:10:56,907 INFO [RS:3;jenkins-hbase4:34917] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-24 20:10:56,907 INFO [RS:3;jenkins-hbase4:34917] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-24 20:10:56,910 INFO [RS:3;jenkins-hbase4:34917] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:56,910 DEBUG [RS:3;jenkins-hbase4:34917] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x01c5fe6f to 127.0.0.1:62881 2023-07-24 20:10:56,910 DEBUG [RS:3;jenkins-hbase4:34917] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:10:56,910 INFO [RS:3;jenkins-hbase4:34917] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-24 20:10:56,910 INFO [RS:3;jenkins-hbase4:34917] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-24 20:10:56,910 INFO [RS:3;jenkins-hbase4:34917] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-24 20:10:56,910 INFO [RS:3;jenkins-hbase4:34917] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-24 20:10:56,912 INFO [RS:3;jenkins-hbase4:34917] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-07-24 20:10:56,912 DEBUG [RS:3;jenkins-hbase4:34917] regionserver.HRegionServer(1478): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-07-24 20:10:56,912 DEBUG [RS:3;jenkins-hbase4:34917] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-07-24 20:10:56,915 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-24 20:10:56,915 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-24 20:10:56,915 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-24 20:10:56,915 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-24 20:10:56,915 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-24 20:10:56,915 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=42.71 KB heapSize=68.52 KB 2023-07-24 20:10:56,920 DEBUG [RS:4;jenkins-hbase4:45209] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/oldWALs 2023-07-24 20:10:56,920 INFO [RS:4;jenkins-hbase4:45209] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C45209%2C1690229445078:(num 1690229445318) 2023-07-24 20:10:56,920 DEBUG [RS:4;jenkins-hbase4:45209] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:10:56,920 INFO [RS:4;jenkins-hbase4:45209] regionserver.LeaseManager(133): Closed leases 2023-07-24 20:10:56,920 INFO [RS:4;jenkins-hbase4:45209] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-24 20:10:56,921 INFO [RS:4;jenkins-hbase4:45209] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-24 20:10:56,921 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-24 20:10:56,921 INFO [RS:4;jenkins-hbase4:45209] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-24 20:10:56,921 INFO [RS:4;jenkins-hbase4:45209] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-24 20:10:56,923 INFO [RS:4;jenkins-hbase4:45209] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:45209 2023-07-24 20:10:56,927 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-24 20:10:56,927 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-24 20:10:56,928 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-24 20:10:56,928 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-24 20:10:56,929 DEBUG [RS:0;jenkins-hbase4:34483] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/oldWALs 2023-07-24 20:10:56,929 INFO [RS:0;jenkins-hbase4:34483] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34483%2C1690229439858:(num 1690229442118) 2023-07-24 20:10:56,929 DEBUG [RS:0;jenkins-hbase4:34483] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:10:56,929 INFO [RS:0;jenkins-hbase4:34483] regionserver.LeaseManager(133): Closed leases 2023-07-24 20:10:56,929 INFO [RS:0;jenkins-hbase4:34483] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-24 20:10:56,929 INFO [RS:0;jenkins-hbase4:34483] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-24 20:10:56,929 INFO [RS:0;jenkins-hbase4:34483] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-24 20:10:56,929 INFO [RS:0;jenkins-hbase4:34483] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-24 20:10:56,930 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-24 20:10:56,931 INFO [RS:0;jenkins-hbase4:34483] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:34483 2023-07-24 20:10:56,959 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=365 B at sequenceid=11 (bloomFilter=true), to=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/namespace/324410cf2a29c2dca1a19e8570799d47/.tmp/info/81b8365c87814c2e9ac1bfab2b6bb254 2023-07-24 20:10:56,979 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=37.25 KB at sequenceid=140 (bloomFilter=false), to=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/.tmp/info/71cc04e9686e4212a8a52f78335cc5c6 2023-07-24 20:10:56,980 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 81b8365c87814c2e9ac1bfab2b6bb254 2023-07-24 20:10:56,983 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/namespace/324410cf2a29c2dca1a19e8570799d47/.tmp/info/81b8365c87814c2e9ac1bfab2b6bb254 as hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/namespace/324410cf2a29c2dca1a19e8570799d47/info/81b8365c87814c2e9ac1bfab2b6bb254 2023-07-24 20:10:56,988 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 71cc04e9686e4212a8a52f78335cc5c6 2023-07-24 20:10:56,992 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 81b8365c87814c2e9ac1bfab2b6bb254 2023-07-24 20:10:56,992 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/namespace/324410cf2a29c2dca1a19e8570799d47/info/81b8365c87814c2e9ac1bfab2b6bb254, entries=5, sequenceid=11, filesize=5.1 K 2023-07-24 20:10:56,994 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~365 B/365, heapSize ~1.11 KB/1136, currentSize=0 B/0 for 324410cf2a29c2dca1a19e8570799d47 in 95ms, sequenceid=11, compaction requested=false 2023-07-24 20:10:57,002 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-07-24 20:10:57,002 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-07-24 20:10:57,006 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/namespace/324410cf2a29c2dca1a19e8570799d47/recovered.edits/14.seqid, newMaxSeqId=14, maxSeqId=1 2023-07-24 20:10:57,007 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. 2023-07-24 20:10:57,007 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 324410cf2a29c2dca1a19e8570799d47: 2023-07-24 20:10:57,007 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. 2023-07-24 20:10:57,010 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ec4f892f95beff2c1aa7863da43a18fa, disabling compactions & flushes 2023-07-24 20:10:57,010 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. 2023-07-24 20:10:57,010 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. 2023-07-24 20:10:57,010 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. after waiting 0 ms 2023-07-24 20:10:57,010 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. 2023-07-24 20:10:57,010 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing ec4f892f95beff2c1aa7863da43a18fa 1/1 column families, dataSize=16.53 KB heapSize=26.77 KB 2023-07-24 20:10:57,016 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:10:57,016 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34917-0x10198f4f3ce000b, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:57,016 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:45209-0x10198f4f3ce000d, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:57,016 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:45209-0x10198f4f3ce000d, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:10:57,016 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:57,016 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:10:57,016 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34917-0x10198f4f3ce000b, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:10:57,017 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:57,017 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:45209-0x10198f4f3ce000d, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:57,017 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,45209,1690229445078] 2023-07-24 20:10:57,017 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,45209,1690229445078; numProcessing=1 2023-07-24 20:10:57,017 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34917-0x10198f4f3ce000b, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:57,018 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,45209,1690229445078 already deleted, retry=false 2023-07-24 20:10:57,018 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,45209,1690229445078 expired; onlineServers=3 2023-07-24 20:10:57,018 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,34483,1690229439858] 2023-07-24 20:10:57,018 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,34483,1690229439858; numProcessing=2 2023-07-24 20:10:57,018 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34483,1690229439858 2023-07-24 20:10:57,019 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:10:57,019 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,45209,1690229445078 2023-07-24 20:10:57,020 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,34483,1690229439858 already deleted, retry=false 2023-07-24 20:10:57,020 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,34483,1690229439858 expired; onlineServers=2 2023-07-24 20:10:57,023 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=1.73 KB at sequenceid=140 (bloomFilter=false), to=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/.tmp/rep_barrier/d4449076caff44088d8e3a03756fcbd6 2023-07-24 20:10:57,031 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d4449076caff44088d8e3a03756fcbd6 2023-07-24 20:10:57,055 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=16.53 KB at sequenceid=67 (bloomFilter=true), to=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/rsgroup/ec4f892f95beff2c1aa7863da43a18fa/.tmp/m/327d57459138467dbba127685af878e9 2023-07-24 20:10:57,064 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 327d57459138467dbba127685af878e9 2023-07-24 20:10:57,065 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/rsgroup/ec4f892f95beff2c1aa7863da43a18fa/.tmp/m/327d57459138467dbba127685af878e9 as hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/rsgroup/ec4f892f95beff2c1aa7863da43a18fa/m/327d57459138467dbba127685af878e9 2023-07-24 20:10:57,071 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=3.73 KB at sequenceid=140 (bloomFilter=false), to=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/.tmp/table/92a06725a50449fa925e17781e43f1e3 2023-07-24 20:10:57,073 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 327d57459138467dbba127685af878e9 2023-07-24 20:10:57,073 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/rsgroup/ec4f892f95beff2c1aa7863da43a18fa/m/327d57459138467dbba127685af878e9, entries=21, sequenceid=67, filesize=5.7 K 2023-07-24 20:10:57,074 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~16.53 KB/16928, heapSize ~26.76 KB/27400, currentSize=0 B/0 for ec4f892f95beff2c1aa7863da43a18fa in 64ms, sequenceid=67, compaction requested=false 2023-07-24 20:10:57,079 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 92a06725a50449fa925e17781e43f1e3 2023-07-24 20:10:57,080 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/.tmp/info/71cc04e9686e4212a8a52f78335cc5c6 as hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/info/71cc04e9686e4212a8a52f78335cc5c6 2023-07-24 20:10:57,081 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/rsgroup/ec4f892f95beff2c1aa7863da43a18fa/recovered.edits/70.seqid, newMaxSeqId=70, maxSeqId=1 2023-07-24 20:10:57,081 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-24 20:10:57,082 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. 2023-07-24 20:10:57,082 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ec4f892f95beff2c1aa7863da43a18fa: 2023-07-24 20:10:57,083 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. 2023-07-24 20:10:57,087 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 71cc04e9686e4212a8a52f78335cc5c6 2023-07-24 20:10:57,087 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/info/71cc04e9686e4212a8a52f78335cc5c6, entries=32, sequenceid=140, filesize=8.2 K 2023-07-24 20:10:57,088 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/.tmp/rep_barrier/d4449076caff44088d8e3a03756fcbd6 as hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/rep_barrier/d4449076caff44088d8e3a03756fcbd6 2023-07-24 20:10:57,094 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d4449076caff44088d8e3a03756fcbd6 2023-07-24 20:10:57,094 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/rep_barrier/d4449076caff44088d8e3a03756fcbd6, entries=16, sequenceid=140, filesize=6.7 K 2023-07-24 20:10:57,095 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/.tmp/table/92a06725a50449fa925e17781e43f1e3 as hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/table/92a06725a50449fa925e17781e43f1e3 2023-07-24 20:10:57,097 INFO [RS:2;jenkins-hbase4:39791] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,39791,1690229440084; all regions closed. 2023-07-24 20:10:57,105 DEBUG [RS:2;jenkins-hbase4:39791] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/oldWALs 2023-07-24 20:10:57,105 INFO [RS:2;jenkins-hbase4:39791] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C39791%2C1690229440084:(num 1690229442118) 2023-07-24 20:10:57,105 DEBUG [RS:2;jenkins-hbase4:39791] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:10:57,105 INFO [RS:2;jenkins-hbase4:39791] regionserver.LeaseManager(133): Closed leases 2023-07-24 20:10:57,106 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 92a06725a50449fa925e17781e43f1e3 2023-07-24 20:10:57,106 INFO [RS:2;jenkins-hbase4:39791] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-24 20:10:57,106 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/table/92a06725a50449fa925e17781e43f1e3, entries=23, sequenceid=140, filesize=7.0 K 2023-07-24 20:10:57,106 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-24 20:10:57,106 INFO [RS:2;jenkins-hbase4:39791] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-24 20:10:57,106 INFO [RS:2;jenkins-hbase4:39791] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-24 20:10:57,106 INFO [RS:2;jenkins-hbase4:39791] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-24 20:10:57,107 INFO [RS:2;jenkins-hbase4:39791] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:39791 2023-07-24 20:10:57,110 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~42.71 KB/43734, heapSize ~68.48 KB/70120, currentSize=0 B/0 for 1588230740 in 195ms, sequenceid=140, compaction requested=false 2023-07-24 20:10:57,113 DEBUG [RS:3;jenkins-hbase4:34917] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-07-24 20:10:57,123 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/recovered.edits/143.seqid, newMaxSeqId=143, maxSeqId=17 2023-07-24 20:10:57,124 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-24 20:10:57,124 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-24 20:10:57,124 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-24 20:10:57,124 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-24 20:10:57,126 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 20:10:57,126 INFO [RS:0;jenkins-hbase4:34483] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,34483,1690229439858; zookeeper connection closed. 2023-07-24 20:10:57,126 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34483-0x10198f4f3ce0001, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 20:10:57,126 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@4213e67b] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@4213e67b 2023-07-24 20:10:57,127 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:57,127 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:10:57,127 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34917-0x10198f4f3ce000b, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,39791,1690229440084 2023-07-24 20:10:57,128 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,39791,1690229440084] 2023-07-24 20:10:57,128 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,39791,1690229440084; numProcessing=3 2023-07-24 20:10:57,130 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,39791,1690229440084 already deleted, retry=false 2023-07-24 20:10:57,130 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,39791,1690229440084 expired; onlineServers=1 2023-07-24 20:10:57,313 INFO [RS:3;jenkins-hbase4:34917] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,34917,1690229443609; all regions closed. 2023-07-24 20:10:57,320 DEBUG [RS:3;jenkins-hbase4:34917] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/oldWALs 2023-07-24 20:10:57,320 INFO [RS:3;jenkins-hbase4:34917] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34917%2C1690229443609.meta:.meta(num 1690229444589) 2023-07-24 20:10:57,327 DEBUG [RS:3;jenkins-hbase4:34917] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/oldWALs 2023-07-24 20:10:57,327 INFO [RS:3;jenkins-hbase4:34917] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34917%2C1690229443609:(num 1690229443883) 2023-07-24 20:10:57,327 DEBUG [RS:3;jenkins-hbase4:34917] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:10:57,327 INFO [RS:3;jenkins-hbase4:34917] regionserver.LeaseManager(133): Closed leases 2023-07-24 20:10:57,328 INFO [RS:3;jenkins-hbase4:34917] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-24 20:10:57,328 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-24 20:10:57,329 INFO [RS:3;jenkins-hbase4:34917] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:34917 2023-07-24 20:10:57,330 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34917-0x10198f4f3ce000b, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34917,1690229443609 2023-07-24 20:10:57,330 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:10:57,331 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,34917,1690229443609] 2023-07-24 20:10:57,331 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,34917,1690229443609; numProcessing=4 2023-07-24 20:10:57,332 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,34917,1690229443609 already deleted, retry=false 2023-07-24 20:10:57,332 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,34917,1690229443609 expired; onlineServers=0 2023-07-24 20:10:57,332 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,39247,1690229437650' ***** 2023-07-24 20:10:57,332 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-24 20:10:57,333 DEBUG [M:0;jenkins-hbase4:39247] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@210a5cb7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-24 20:10:57,333 INFO [M:0;jenkins-hbase4:39247] regionserver.HRegionServer(1109): Stopping infoServer 2023-07-24 20:10:57,335 INFO [M:0;jenkins-hbase4:39247] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.w.WebAppContext@12930a{master,/,null,STOPPED}{file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/master} 2023-07-24 20:10:57,335 INFO [M:0;jenkins-hbase4:39247] server.AbstractConnector(383): Stopped ServerConnector@309e9f1{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-24 20:10:57,335 INFO [M:0;jenkins-hbase4:39247] session.HouseKeeper(149): node0 Stopped scavenging 2023-07-24 20:10:57,335 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-07-24 20:10:57,336 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 20:10:57,336 INFO [M:0;jenkins-hbase4:39247] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@562b38c7{static,/static,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/static/,STOPPED} 2023-07-24 20:10:57,336 INFO [M:0;jenkins-hbase4:39247] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@45ca76be{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/hadoop.log.dir/,STOPPED} 2023-07-24 20:10:57,337 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-24 20:10:57,337 INFO [M:0;jenkins-hbase4:39247] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,39247,1690229437650 2023-07-24 20:10:57,337 INFO [M:0;jenkins-hbase4:39247] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,39247,1690229437650; all regions closed. 2023-07-24 20:10:57,337 DEBUG [M:0;jenkins-hbase4:39247] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:10:57,337 INFO [M:0;jenkins-hbase4:39247] master.HMaster(1491): Stopping master jetty server 2023-07-24 20:10:57,338 INFO [M:0;jenkins-hbase4:39247] server.AbstractConnector(383): Stopped ServerConnector@4545c88a{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-24 20:10:57,338 DEBUG [M:0;jenkins-hbase4:39247] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-24 20:10:57,338 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-24 20:10:57,338 DEBUG [M:0;jenkins-hbase4:39247] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-24 20:10:57,338 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1690229441686] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1690229441686,5,FailOnTimeoutGroup] 2023-07-24 20:10:57,338 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1690229441687] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1690229441687,5,FailOnTimeoutGroup] 2023-07-24 20:10:57,338 INFO [M:0;jenkins-hbase4:39247] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-24 20:10:57,338 INFO [M:0;jenkins-hbase4:39247] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-24 20:10:57,339 INFO [M:0;jenkins-hbase4:39247] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-24 20:10:57,339 DEBUG [M:0;jenkins-hbase4:39247] master.HMaster(1512): Stopping service threads 2023-07-24 20:10:57,339 INFO [M:0;jenkins-hbase4:39247] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-24 20:10:57,339 ERROR [M:0;jenkins-hbase4:39247] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-07-24 20:10:57,340 INFO [M:0;jenkins-hbase4:39247] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 20:10:57,340 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-24 20:10:57,340 DEBUG [M:0;jenkins-hbase4:39247] zookeeper.ZKUtil(398): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-07-24 20:10:57,340 WARN [M:0;jenkins-hbase4:39247] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-24 20:10:57,340 INFO [M:0;jenkins-hbase4:39247] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 20:10:57,340 INFO [M:0;jenkins-hbase4:39247] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-24 20:10:57,340 DEBUG [M:0;jenkins-hbase4:39247] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-24 20:10:57,341 INFO [M:0;jenkins-hbase4:39247] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-24 20:10:57,341 DEBUG [M:0;jenkins-hbase4:39247] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-24 20:10:57,341 DEBUG [M:0;jenkins-hbase4:39247] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-24 20:10:57,341 DEBUG [M:0;jenkins-hbase4:39247] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-24 20:10:57,341 INFO [M:0;jenkins-hbase4:39247] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=363.53 KB heapSize=433.15 KB 2023-07-24 20:10:57,363 INFO [M:0;jenkins-hbase4:39247] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=363.53 KB at sequenceid=796 (bloomFilter=true), to=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/b21deb618bb94cbcaa2b660c8c643e86 2023-07-24 20:10:57,371 DEBUG [M:0;jenkins-hbase4:39247] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/b21deb618bb94cbcaa2b660c8c643e86 as hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/b21deb618bb94cbcaa2b660c8c643e86 2023-07-24 20:10:57,378 INFO [M:0;jenkins-hbase4:39247] regionserver.HStore(1080): Added hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/b21deb618bb94cbcaa2b660c8c643e86, entries=108, sequenceid=796, filesize=25.2 K 2023-07-24 20:10:57,379 INFO [M:0;jenkins-hbase4:39247] regionserver.HRegion(2948): Finished flush of dataSize ~363.53 KB/372253, heapSize ~433.13 KB/443528, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 38ms, sequenceid=796, compaction requested=false 2023-07-24 20:10:57,381 INFO [M:0;jenkins-hbase4:39247] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-24 20:10:57,381 DEBUG [M:0;jenkins-hbase4:39247] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-24 20:10:57,386 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-24 20:10:57,386 INFO [M:0;jenkins-hbase4:39247] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-24 20:10:57,387 INFO [M:0;jenkins-hbase4:39247] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:39247 2023-07-24 20:10:57,389 DEBUG [M:0;jenkins-hbase4:39247] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,39247,1690229437650 already deleted, retry=false 2023-07-24 20:10:57,472 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34917-0x10198f4f3ce000b, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 20:10:57,472 INFO [RS:3;jenkins-hbase4:34917] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,34917,1690229443609; zookeeper connection closed. 2023-07-24 20:10:57,472 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34917-0x10198f4f3ce000b, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 20:10:57,472 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@1f763044] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@1f763044 2023-07-24 20:10:57,572 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 20:10:57,572 INFO [RS:2;jenkins-hbase4:39791] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,39791,1690229440084; zookeeper connection closed. 2023-07-24 20:10:57,572 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:39791-0x10198f4f3ce0003, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 20:10:57,573 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@2744fe63] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@2744fe63 2023-07-24 20:10:57,672 INFO [RS:4;jenkins-hbase4:45209] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,45209,1690229445078; zookeeper connection closed. 2023-07-24 20:10:57,673 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:45209-0x10198f4f3ce000d, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 20:10:57,673 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:45209-0x10198f4f3ce000d, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 20:10:57,673 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@72e3c597] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@72e3c597 2023-07-24 20:10:57,673 INFO [Listener at localhost/44145] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 5 regionserver(s) complete 2023-07-24 20:10:57,773 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 20:10:57,773 INFO [M:0;jenkins-hbase4:39247] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,39247,1690229437650; zookeeper connection closed. 2023-07-24 20:10:57,773 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:39247-0x10198f4f3ce0000, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 20:10:57,775 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBasics(311): Sleeping a bit 2023-07-24 20:10:57,892 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-24 20:10:57,892 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: Master,sub=Coprocessor.Master.CP_org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase$CPMasterObserver 2023-07-24 20:10:57,892 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: Master,sub=Coprocessor.Master.CP_org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint 2023-07-24 20:10:59,419 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-24 20:10:59,777 DEBUG [Listener at localhost/44145] hbase.LocalHBaseCluster(134): Setting Master Port to random. 2023-07-24 20:10:59,777 DEBUG [Listener at localhost/44145] hbase.LocalHBaseCluster(141): Setting RegionServer Port to random. 2023-07-24 20:10:59,777 DEBUG [Listener at localhost/44145] hbase.LocalHBaseCluster(151): Setting RS InfoServer Port to random. 2023-07-24 20:10:59,777 DEBUG [Listener at localhost/44145] hbase.LocalHBaseCluster(159): Setting Master InfoServer Port to random. 2023-07-24 20:10:59,778 INFO [Listener at localhost/44145] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-07-24 20:10:59,779 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 20:10:59,779 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-24 20:10:59,779 INFO [Listener at localhost/44145] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-24 20:10:59,779 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 20:10:59,779 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-24 20:10:59,779 INFO [Listener at localhost/44145] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-24 20:10:59,783 INFO [Listener at localhost/44145] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33409 2023-07-24 20:10:59,784 INFO [Listener at localhost/44145] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:10:59,785 INFO [Listener at localhost/44145] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:10:59,786 INFO [Listener at localhost/44145] zookeeper.RecoverableZooKeeper(93): Process identifier=master:33409 connecting to ZooKeeper ensemble=127.0.0.1:62881 2023-07-24 20:10:59,794 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): master:334090x0, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-24 20:10:59,794 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): master:334090x0, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 20:10:59,794 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:334090x0, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-24 20:10:59,795 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): master:334090x0, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-24 20:10:59,808 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:33409-0x10198f4f3ce0010 connected 2023-07-24 20:10:59,811 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33409 2023-07-24 20:10:59,812 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33409 2023-07-24 20:10:59,814 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33409 2023-07-24 20:10:59,827 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33409 2023-07-24 20:10:59,828 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33409 2023-07-24 20:10:59,831 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2023-07-24 20:10:59,831 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter) 2023-07-24 20:10:59,831 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'securityheaders' (class=org.apache.hadoop.hbase.http.SecurityHeadersFilter) 2023-07-24 20:10:59,832 INFO [Listener at localhost/44145] http.HttpServer(879): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context master 2023-07-24 20:10:59,832 INFO [Listener at localhost/44145] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2023-07-24 20:10:59,832 INFO [Listener at localhost/44145] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2023-07-24 20:10:59,832 INFO [Listener at localhost/44145] http.HttpServer(783): ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint. 2023-07-24 20:10:59,834 INFO [Listener at localhost/44145] http.HttpServer(1146): Jetty bound to port 37241 2023-07-24 20:10:59,834 INFO [Listener at localhost/44145] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-24 20:10:59,843 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:10:59,843 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@fa245c3{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/hadoop.log.dir/,AVAILABLE} 2023-07-24 20:10:59,844 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:10:59,844 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@726dac4c{static,/static,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/static/,AVAILABLE} 2023-07-24 20:10:59,853 INFO [Listener at localhost/44145] webapp.StandardDescriptorProcessor(277): NO JSP Support for /, did not find org.apache.hbase.thirdparty.org.eclipse.jetty.jsp.JettyJspServlet 2023-07-24 20:10:59,854 INFO [Listener at localhost/44145] session.DefaultSessionIdManager(334): DefaultSessionIdManager workerName=node0 2023-07-24 20:10:59,854 INFO [Listener at localhost/44145] session.DefaultSessionIdManager(339): No SessionScavenger set, using defaults 2023-07-24 20:10:59,854 INFO [Listener at localhost/44145] session.HouseKeeper(132): node0 Scavenging every 660000ms 2023-07-24 20:10:59,856 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:10:59,858 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.w.WebAppContext@1f8ba06{master,/,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/master/,AVAILABLE}{file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/master} 2023-07-24 20:10:59,859 INFO [Listener at localhost/44145] server.AbstractConnector(333): Started ServerConnector@58b84ff5{HTTP/1.1, (http/1.1)}{0.0.0.0:37241} 2023-07-24 20:10:59,859 INFO [Listener at localhost/44145] server.Server(415): Started @28427ms 2023-07-24 20:10:59,860 INFO [Listener at localhost/44145] master.HMaster(444): hbase.rootdir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4, hbase.cluster.distributed=false 2023-07-24 20:10:59,862 DEBUG [pool-349-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: INIT 2023-07-24 20:10:59,880 INFO [Listener at localhost/44145] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-24 20:10:59,881 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 20:10:59,881 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-24 20:10:59,881 INFO [Listener at localhost/44145] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-24 20:10:59,881 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 20:10:59,881 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-24 20:10:59,882 INFO [Listener at localhost/44145] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-24 20:10:59,882 INFO [Listener at localhost/44145] ipc.NettyRpcServer(120): Bind to /172.31.14.131:46705 2023-07-24 20:10:59,883 INFO [Listener at localhost/44145] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-24 20:10:59,884 DEBUG [Listener at localhost/44145] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-24 20:10:59,885 INFO [Listener at localhost/44145] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:10:59,886 INFO [Listener at localhost/44145] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:10:59,887 INFO [Listener at localhost/44145] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:46705 connecting to ZooKeeper ensemble=127.0.0.1:62881 2023-07-24 20:10:59,891 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:467050x0, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-24 20:10:59,892 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): regionserver:467050x0, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-24 20:10:59,894 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:46705-0x10198f4f3ce0011 connected 2023-07-24 20:10:59,894 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): regionserver:46705-0x10198f4f3ce0011, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 20:10:59,894 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): regionserver:46705-0x10198f4f3ce0011, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-24 20:10:59,896 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46705 2023-07-24 20:10:59,899 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46705 2023-07-24 20:10:59,900 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46705 2023-07-24 20:10:59,902 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46705 2023-07-24 20:10:59,902 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46705 2023-07-24 20:10:59,904 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2023-07-24 20:10:59,904 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter) 2023-07-24 20:10:59,904 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'securityheaders' (class=org.apache.hadoop.hbase.http.SecurityHeadersFilter) 2023-07-24 20:10:59,905 INFO [Listener at localhost/44145] http.HttpServer(879): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context regionserver 2023-07-24 20:10:59,905 INFO [Listener at localhost/44145] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2023-07-24 20:10:59,905 INFO [Listener at localhost/44145] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2023-07-24 20:10:59,905 INFO [Listener at localhost/44145] http.HttpServer(783): ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint. 2023-07-24 20:10:59,906 INFO [Listener at localhost/44145] http.HttpServer(1146): Jetty bound to port 45285 2023-07-24 20:10:59,906 INFO [Listener at localhost/44145] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-24 20:10:59,910 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:10:59,911 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@14daa73{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/hadoop.log.dir/,AVAILABLE} 2023-07-24 20:10:59,911 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:10:59,911 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@1a6b043d{static,/static,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/static/,AVAILABLE} 2023-07-24 20:10:59,916 INFO [Listener at localhost/44145] webapp.StandardDescriptorProcessor(277): NO JSP Support for /, did not find org.apache.hbase.thirdparty.org.eclipse.jetty.jsp.JettyJspServlet 2023-07-24 20:10:59,917 INFO [Listener at localhost/44145] session.DefaultSessionIdManager(334): DefaultSessionIdManager workerName=node0 2023-07-24 20:10:59,917 INFO [Listener at localhost/44145] session.DefaultSessionIdManager(339): No SessionScavenger set, using defaults 2023-07-24 20:10:59,918 INFO [Listener at localhost/44145] session.HouseKeeper(132): node0 Scavenging every 600000ms 2023-07-24 20:10:59,918 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:10:59,919 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.w.WebAppContext@722a21a7{regionserver,/,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver/,AVAILABLE}{file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver} 2023-07-24 20:10:59,920 INFO [Listener at localhost/44145] server.AbstractConnector(333): Started ServerConnector@f3c8deb{HTTP/1.1, (http/1.1)}{0.0.0.0:45285} 2023-07-24 20:10:59,920 INFO [Listener at localhost/44145] server.Server(415): Started @28488ms 2023-07-24 20:10:59,932 INFO [Listener at localhost/44145] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-24 20:10:59,932 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 20:10:59,932 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-24 20:10:59,932 INFO [Listener at localhost/44145] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-24 20:10:59,932 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 20:10:59,932 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-24 20:10:59,932 INFO [Listener at localhost/44145] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-24 20:10:59,933 INFO [Listener at localhost/44145] ipc.NettyRpcServer(120): Bind to /172.31.14.131:45019 2023-07-24 20:10:59,933 INFO [Listener at localhost/44145] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-24 20:10:59,935 DEBUG [Listener at localhost/44145] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-24 20:10:59,935 INFO [Listener at localhost/44145] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:10:59,936 INFO [Listener at localhost/44145] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:10:59,937 INFO [Listener at localhost/44145] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:45019 connecting to ZooKeeper ensemble=127.0.0.1:62881 2023-07-24 20:10:59,942 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:450190x0, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-24 20:10:59,945 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): regionserver:450190x0, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-24 20:10:59,946 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): regionserver:450190x0, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 20:10:59,946 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:45019-0x10198f4f3ce0012 connected 2023-07-24 20:10:59,947 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): regionserver:45019-0x10198f4f3ce0012, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-24 20:10:59,961 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=45019 2023-07-24 20:10:59,961 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=45019 2023-07-24 20:10:59,974 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=45019 2023-07-24 20:10:59,988 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=45019 2023-07-24 20:10:59,992 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=45019 2023-07-24 20:10:59,995 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2023-07-24 20:10:59,995 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter) 2023-07-24 20:10:59,999 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'securityheaders' (class=org.apache.hadoop.hbase.http.SecurityHeadersFilter) 2023-07-24 20:10:59,999 INFO [Listener at localhost/44145] http.HttpServer(879): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context regionserver 2023-07-24 20:10:59,999 INFO [Listener at localhost/44145] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2023-07-24 20:10:59,999 INFO [Listener at localhost/44145] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2023-07-24 20:11:00,000 INFO [Listener at localhost/44145] http.HttpServer(783): ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint. 2023-07-24 20:11:00,000 INFO [Listener at localhost/44145] http.HttpServer(1146): Jetty bound to port 37247 2023-07-24 20:11:00,000 INFO [Listener at localhost/44145] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-24 20:11:00,034 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:11:00,034 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@38e22bc7{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/hadoop.log.dir/,AVAILABLE} 2023-07-24 20:11:00,034 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:11:00,035 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@53cc2d4e{static,/static,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/static/,AVAILABLE} 2023-07-24 20:11:00,040 INFO [Listener at localhost/44145] webapp.StandardDescriptorProcessor(277): NO JSP Support for /, did not find org.apache.hbase.thirdparty.org.eclipse.jetty.jsp.JettyJspServlet 2023-07-24 20:11:00,041 INFO [Listener at localhost/44145] session.DefaultSessionIdManager(334): DefaultSessionIdManager workerName=node0 2023-07-24 20:11:00,041 INFO [Listener at localhost/44145] session.DefaultSessionIdManager(339): No SessionScavenger set, using defaults 2023-07-24 20:11:00,041 INFO [Listener at localhost/44145] session.HouseKeeper(132): node0 Scavenging every 600000ms 2023-07-24 20:11:00,046 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:11:00,047 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.w.WebAppContext@55838389{regionserver,/,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver/,AVAILABLE}{file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver} 2023-07-24 20:11:00,049 INFO [Listener at localhost/44145] server.AbstractConnector(333): Started ServerConnector@64a829ad{HTTP/1.1, (http/1.1)}{0.0.0.0:37247} 2023-07-24 20:11:00,049 INFO [Listener at localhost/44145] server.Server(415): Started @28616ms 2023-07-24 20:11:00,063 INFO [Listener at localhost/44145] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-24 20:11:00,065 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 20:11:00,065 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-24 20:11:00,065 INFO [Listener at localhost/44145] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-24 20:11:00,065 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 20:11:00,065 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-24 20:11:00,065 INFO [Listener at localhost/44145] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-24 20:11:00,073 INFO [Listener at localhost/44145] ipc.NettyRpcServer(120): Bind to /172.31.14.131:46065 2023-07-24 20:11:00,073 INFO [Listener at localhost/44145] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-24 20:11:00,104 DEBUG [Listener at localhost/44145] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-24 20:11:00,105 INFO [Listener at localhost/44145] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:11:00,106 INFO [Listener at localhost/44145] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:11:00,107 INFO [Listener at localhost/44145] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:46065 connecting to ZooKeeper ensemble=127.0.0.1:62881 2023-07-24 20:11:00,112 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:460650x0, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-24 20:11:00,113 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): regionserver:460650x0, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-24 20:11:00,114 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:46065-0x10198f4f3ce0013 connected 2023-07-24 20:11:00,115 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): regionserver:46065-0x10198f4f3ce0013, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 20:11:00,115 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): regionserver:46065-0x10198f4f3ce0013, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-24 20:11:00,116 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46065 2023-07-24 20:11:00,116 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46065 2023-07-24 20:11:00,117 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46065 2023-07-24 20:11:00,118 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46065 2023-07-24 20:11:00,119 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46065 2023-07-24 20:11:00,120 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2023-07-24 20:11:00,121 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter) 2023-07-24 20:11:00,121 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'securityheaders' (class=org.apache.hadoop.hbase.http.SecurityHeadersFilter) 2023-07-24 20:11:00,121 INFO [Listener at localhost/44145] http.HttpServer(879): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context regionserver 2023-07-24 20:11:00,121 INFO [Listener at localhost/44145] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2023-07-24 20:11:00,121 INFO [Listener at localhost/44145] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2023-07-24 20:11:00,121 INFO [Listener at localhost/44145] http.HttpServer(783): ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint. 2023-07-24 20:11:00,122 INFO [Listener at localhost/44145] http.HttpServer(1146): Jetty bound to port 37885 2023-07-24 20:11:00,122 INFO [Listener at localhost/44145] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-24 20:11:00,123 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:11:00,123 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@606f4bdb{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/hadoop.log.dir/,AVAILABLE} 2023-07-24 20:11:00,123 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:11:00,124 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@7f5a1268{static,/static,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/static/,AVAILABLE} 2023-07-24 20:11:00,128 INFO [Listener at localhost/44145] webapp.StandardDescriptorProcessor(277): NO JSP Support for /, did not find org.apache.hbase.thirdparty.org.eclipse.jetty.jsp.JettyJspServlet 2023-07-24 20:11:00,129 INFO [Listener at localhost/44145] session.DefaultSessionIdManager(334): DefaultSessionIdManager workerName=node0 2023-07-24 20:11:00,129 INFO [Listener at localhost/44145] session.DefaultSessionIdManager(339): No SessionScavenger set, using defaults 2023-07-24 20:11:00,129 INFO [Listener at localhost/44145] session.HouseKeeper(132): node0 Scavenging every 600000ms 2023-07-24 20:11:00,130 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:11:00,131 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.w.WebAppContext@68e950a2{regionserver,/,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver/,AVAILABLE}{file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver} 2023-07-24 20:11:00,133 INFO [Listener at localhost/44145] server.AbstractConnector(333): Started ServerConnector@63c9e904{HTTP/1.1, (http/1.1)}{0.0.0.0:37885} 2023-07-24 20:11:00,133 INFO [Listener at localhost/44145] server.Server(415): Started @28700ms 2023-07-24 20:11:00,134 INFO [master/jenkins-hbase4:0:becomeActiveMaster] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-24 20:11:00,138 INFO [master/jenkins-hbase4:0:becomeActiveMaster] server.AbstractConnector(333): Started ServerConnector@194f0464{HTTP/1.1, (http/1.1)}{0.0.0.0:39189} 2023-07-24 20:11:00,138 INFO [master/jenkins-hbase4:0:becomeActiveMaster] server.Server(415): Started @28706ms 2023-07-24 20:11:00,139 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,33409,1690229459778 2023-07-24 20:11:00,140 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-07-24 20:11:00,140 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,33409,1690229459778 2023-07-24 20:11:00,143 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-24 20:11:00,143 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:46705-0x10198f4f3ce0011, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-24 20:11:00,143 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:46065-0x10198f4f3ce0013, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-24 20:11:00,143 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 20:11:00,143 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:45019-0x10198f4f3ce0012, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-24 20:11:00,146 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-24 20:11:00,147 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-24 20:11:00,147 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,33409,1690229459778 from backup master directory 2023-07-24 20:11:00,148 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,33409,1690229459778 2023-07-24 20:11:00,148 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-07-24 20:11:00,148 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-24 20:11:00,148 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,33409,1690229459778 2023-07-24 20:11:00,171 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:11:00,198 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x4d670eec to 127.0.0.1:62881 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-24 20:11:00,203 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5192c18e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-24 20:11:00,203 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-24 20:11:00,204 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-24 20:11:00,204 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-24 20:11:00,211 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/WALs/jenkins-hbase4.apache.org,39247,1690229437650 to hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/WALs/jenkins-hbase4.apache.org,39247,1690229437650-dead as it is dead 2023-07-24 20:11:00,213 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/WALs/jenkins-hbase4.apache.org,39247,1690229437650-dead/jenkins-hbase4.apache.org%2C39247%2C1690229437650.1690229440797 2023-07-24 20:11:00,217 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/WALs/jenkins-hbase4.apache.org,39247,1690229437650-dead/jenkins-hbase4.apache.org%2C39247%2C1690229437650.1690229440797 after 4ms 2023-07-24 20:11:00,218 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/WALs/jenkins-hbase4.apache.org,39247,1690229437650-dead/jenkins-hbase4.apache.org%2C39247%2C1690229437650.1690229440797 to hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C39247%2C1690229437650.1690229440797 2023-07-24 20:11:00,218 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/WALs/jenkins-hbase4.apache.org,39247,1690229437650-dead 2023-07-24 20:11:00,218 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/WALs/jenkins-hbase4.apache.org,33409,1690229459778 2023-07-24 20:11:00,221 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C33409%2C1690229459778, suffix=, logDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/WALs/jenkins-hbase4.apache.org,33409,1690229459778, archiveDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/oldWALs, maxLogs=10 2023-07-24 20:11:00,247 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK] 2023-07-24 20:11:00,248 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK] 2023-07-24 20:11:00,250 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK] 2023-07-24 20:11:00,256 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/WALs/jenkins-hbase4.apache.org,33409,1690229459778/jenkins-hbase4.apache.org%2C33409%2C1690229459778.1690229460221 2023-07-24 20:11:00,257 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK], DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK], DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK]] 2023-07-24 20:11:00,257 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-24 20:11:00,257 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:11:00,257 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-24 20:11:00,257 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-24 20:11:00,259 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-24 20:11:00,260 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-24 20:11:00,261 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-24 20:11:00,268 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/b21deb618bb94cbcaa2b660c8c643e86 2023-07-24 20:11:00,268 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:11:00,269 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-07-24 20:11:00,269 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C39247%2C1690229437650.1690229440797 2023-07-24 20:11:00,308 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 939, firstSequenceIdInLog=3, maxSequenceIdInLog=798, path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C39247%2C1690229437650.1690229440797 2023-07-24 20:11:00,310 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C39247%2C1690229437650.1690229440797 2023-07-24 20:11:00,314 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-24 20:11:00,317 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/798.seqid, newMaxSeqId=798, maxSeqId=1 2023-07-24 20:11:00,318 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=799; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=11376020480, jitterRate=0.05947446823120117}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 20:11:00,318 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-24 20:11:00,318 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-24 20:11:00,319 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-24 20:11:00,319 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-24 20:11:00,320 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 20:11:00,320 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 20:11:00,331 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 20:11:00,332 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 20:11:00,332 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=6, state=SUCCESS; CreateTableProcedure table=hbase:rsgroup 2023-07-24 20:11:00,332 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=10, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 20:11:00,332 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=11, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 20:11:00,333 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, REOPEN/MOVE 2023-07-24 20:11:00,333 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=15, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,34371,1690229440021, splitWal=true, meta=false 2023-07-24 20:11:00,333 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=16, state=SUCCESS; ModifyNamespaceProcedure, namespace=default 2023-07-24 20:11:00,334 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=17, state=SUCCESS; CreateTableProcedure table=Group_testCreateAndAssign 2023-07-24 20:11:00,334 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=20, state=SUCCESS; DisableTableProcedure table=Group_testCreateAndAssign 2023-07-24 20:11:00,334 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=23, state=SUCCESS; DeleteTableProcedure table=Group_testCreateAndAssign 2023-07-24 20:11:00,334 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; CreateTableProcedure table=Group_testCreateMultiRegion 2023-07-24 20:11:00,335 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=45, state=SUCCESS; DisableTableProcedure table=Group_testCreateMultiRegion 2023-07-24 20:11:00,335 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=66, state=SUCCESS; DeleteTableProcedure table=Group_testCreateMultiRegion 2023-07-24 20:11:00,335 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=67, state=SUCCESS; CreateNamespaceProcedure, namespace=Group_foo 2023-07-24 20:11:00,335 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; CreateTableProcedure table=Group_foo:Group_testCreateAndAssign 2023-07-24 20:11:00,335 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=71, state=SUCCESS; DisableTableProcedure table=Group_foo:Group_testCreateAndAssign 2023-07-24 20:11:00,335 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=74, state=SUCCESS; DeleteTableProcedure table=Group_foo:Group_testCreateAndAssign 2023-07-24 20:11:00,336 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=75, state=SUCCESS; DeleteNamespaceProcedure, namespace=Group_foo 2023-07-24 20:11:00,336 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=76, state=SUCCESS; CreateTableProcedure table=Group_testCreateAndDrop 2023-07-24 20:11:00,336 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=Group_testCreateAndDrop 2023-07-24 20:11:00,336 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=82, state=SUCCESS; DeleteTableProcedure table=Group_testCreateAndDrop 2023-07-24 20:11:00,337 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=83, state=SUCCESS; CreateTableProcedure table=Group_testCloneSnapshot 2023-07-24 20:11:00,337 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=86, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=Group_testCloneSnapshot, type=EXCLUSIVE 2023-07-24 20:11:00,337 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=87, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=Group_testCloneSnapshot, type=SHARED 2023-07-24 20:11:00,337 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; CloneSnapshotProcedure (table=Group_testCloneSnapshot_clone snapshot=name: "Group_testCloneSnapshot_snap" table: "Group_testCloneSnapshot" creation_time: 1690229452851 type: FLUSH version: 2 ttl: 0 ) 2023-07-24 20:11:00,338 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=91, state=SUCCESS; DisableTableProcedure table=Group_testCloneSnapshot 2023-07-24 20:11:00,338 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=94, state=SUCCESS; DeleteTableProcedure table=Group_testCloneSnapshot 2023-07-24 20:11:00,338 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=95, state=SUCCESS; DisableTableProcedure table=Group_testCloneSnapshot_clone 2023-07-24 20:11:00,338 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=98, state=SUCCESS; DeleteTableProcedure table=Group_testCloneSnapshot_clone 2023-07-24 20:11:00,338 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=99, state=SUCCESS; CreateNamespaceProcedure, namespace=Group_ns 2023-07-24 20:11:00,339 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=100, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-create-table:org.apache.hadoop.hbase.HBaseIOException: No online servers in the rsgroup appInfo which table Group_ns:testCreateWhenRsgroupNoOnlineServers belongs to; CreateTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-24 20:11:00,339 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; CreateTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-24 20:11:00,339 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=104, state=SUCCESS; DisableTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-24 20:11:00,339 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=107, state=SUCCESS; DeleteTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-24 20:11:00,339 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteNamespaceProcedure, namespace=Group_ns 2023-07-24 20:11:00,340 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 19 msec 2023-07-24 20:11:00,340 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-24 20:11:00,341 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 20:11:00,341 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,34917,1690229443609, table=hbase:meta, region=1588230740 2023-07-24 20:11:00,344 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 4 possibly 'live' servers, and 0 'splitting'. 2023-07-24 20:11:00,349 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,39791,1690229440084 already deleted, retry=false 2023-07-24 20:11:00,349 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,39791,1690229440084 on jenkins-hbase4.apache.org,33409,1690229459778 2023-07-24 20:11:00,350 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,39791,1690229440084, splitWal=true, meta=false 2023-07-24 20:11:00,350 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=109 for jenkins-hbase4.apache.org,39791,1690229440084 (carryingMeta=false) jenkins-hbase4.apache.org,39791,1690229440084/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@2a6d3fbe[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-24 20:11:00,351 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,45209,1690229445078 already deleted, retry=false 2023-07-24 20:11:00,351 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,45209,1690229445078 on jenkins-hbase4.apache.org,33409,1690229459778 2023-07-24 20:11:00,352 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=110, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,45209,1690229445078, splitWal=true, meta=false 2023-07-24 20:11:00,352 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=110 for jenkins-hbase4.apache.org,45209,1690229445078 (carryingMeta=false) jenkins-hbase4.apache.org,45209,1690229445078/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@ab92669[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-24 20:11:00,353 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,34483,1690229439858 already deleted, retry=false 2023-07-24 20:11:00,353 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,34483,1690229439858 on jenkins-hbase4.apache.org,33409,1690229459778 2023-07-24 20:11:00,354 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=111, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,34483,1690229439858, splitWal=true, meta=false 2023-07-24 20:11:00,354 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=111 for jenkins-hbase4.apache.org,34483,1690229439858 (carryingMeta=false) jenkins-hbase4.apache.org,34483,1690229439858/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@4dfc7df0[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-24 20:11:00,355 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,34917,1690229443609 already deleted, retry=false 2023-07-24 20:11:00,355 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,34917,1690229443609 on jenkins-hbase4.apache.org,33409,1690229459778 2023-07-24 20:11:00,356 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,34917,1690229443609, splitWal=true, meta=true 2023-07-24 20:11:00,356 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=112 for jenkins-hbase4.apache.org,34917,1690229443609 (carryingMeta=true) jenkins-hbase4.apache.org,34917,1690229443609/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@2126a0ae[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-24 20:11:00,356 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/balancer 2023-07-24 20:11:00,357 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-24 20:11:00,357 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-07-24 20:11:00,358 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-07-24 20:11:00,359 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-07-24 20:11:00,360 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-07-24 20:11:00,363 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:46705-0x10198f4f3ce0011, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-24 20:11:00,363 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-24 20:11:00,363 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:46065-0x10198f4f3ce0013, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-24 20:11:00,363 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:45019-0x10198f4f3ce0012, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-24 20:11:00,363 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 20:11:00,363 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,33409,1690229459778, sessionid=0x10198f4f3ce0010, setting cluster-up flag (Was=false) 2023-07-24 20:11:00,367 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-07-24 20:11:00,368 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,33409,1690229459778 2023-07-24 20:11:00,371 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-07-24 20:11:00,372 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,33409,1690229459778 2023-07-24 20:11:00,375 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2930): Registered master coprocessor service: service=RSGroupAdminService 2023-07-24 20:11:00,375 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] rsgroup.RSGroupInfoManagerImpl(537): Refreshing in Offline mode. 2023-07-24 20:11:00,377 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] rsgroup.RSGroupInfoManagerImpl(511): Read ZK GroupInfo count:2 2023-07-24 20:11:00,377 INFO [master/jenkins-hbase4:0:becomeActiveMaster] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint loaded, priority=536870911. 2023-07-24 20:11:00,378 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33409,1690229459778] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(787): Updating default servers. 2023-07-24 20:11:00,378 INFO [master/jenkins-hbase4:0:becomeActiveMaster] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase$CPMasterObserver loaded, priority=536870912. 2023-07-24 20:11:00,378 INFO [master/jenkins-hbase4:0:becomeActiveMaster] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.quotas.MasterQuotasObserver loaded, priority=536870913. 2023-07-24 20:11:00,382 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33409,1690229459778] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-24 20:11:00,383 WARN [RS-EventLoopGroup-12-3] ipc.NettyRpcConnection$2(294): Exception encountered while connecting to the server jenkins-hbase4.apache.org/172.31.14.131:34917 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:34917 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:155) at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:128) at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-07-24 20:11:00,384 DEBUG [RS-EventLoopGroup-12-3] ipc.FailedServers(52): Added failed server with address jenkins-hbase4.apache.org/172.31.14.131:34917 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:34917 2023-07-24 20:11:00,393 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-24 20:11:00,393 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-24 20:11:00,393 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-24 20:11:00,393 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-24 20:11:00,393 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-24 20:11:00,393 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-24 20:11:00,393 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-24 20:11:00,393 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-24 20:11:00,393 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-24 20:11:00,393 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:00,393 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-24 20:11:00,393 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:00,395 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690229490395 2023-07-24 20:11:00,397 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-24 20:11:00,397 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-24 20:11:00,397 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-24 20:11:00,398 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-24 20:11:00,398 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-24 20:11:00,398 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-24 20:11:00,398 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:00,399 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase4.apache.org,34917,1690229443609; numProcessing=1 2023-07-24 20:11:00,399 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-24 20:11:00,399 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=112, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,34917,1690229443609, splitWal=true, meta=true 2023-07-24 20:11:00,399 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-24 20:11:00,399 DEBUG [PEWorker-2] master.DeadServer(103): Processing jenkins-hbase4.apache.org,39791,1690229440084; numProcessing=2 2023-07-24 20:11:00,399 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-24 20:11:00,399 DEBUG [PEWorker-4] master.DeadServer(103): Processing jenkins-hbase4.apache.org,34483,1690229439858; numProcessing=3 2023-07-24 20:11:00,399 INFO [PEWorker-2] procedure.ServerCrashProcedure(161): Start pid=109, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,39791,1690229440084, splitWal=true, meta=false 2023-07-24 20:11:00,399 DEBUG [PEWorker-3] master.DeadServer(103): Processing jenkins-hbase4.apache.org,45209,1690229445078; numProcessing=4 2023-07-24 20:11:00,399 INFO [PEWorker-4] procedure.ServerCrashProcedure(161): Start pid=111, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,34483,1690229439858, splitWal=true, meta=false 2023-07-24 20:11:00,399 INFO [PEWorker-3] procedure.ServerCrashProcedure(161): Start pid=110, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,45209,1690229445078, splitWal=true, meta=false 2023-07-24 20:11:00,400 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-24 20:11:00,400 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-24 20:11:00,400 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=112, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,34917,1690229443609, splitWal=true, meta=true, isMeta: true 2023-07-24 20:11:00,400 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1690229460400,5,FailOnTimeoutGroup] 2023-07-24 20:11:00,400 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1690229460400,5,FailOnTimeoutGroup] 2023-07-24 20:11:00,401 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:00,401 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-24 20:11:00,401 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:00,401 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:00,401 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1690229460401, completionTime=-1 2023-07-24 20:11:00,401 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-07-24 20:11:00,401 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-24 20:11:00,402 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,34917,1690229443609-splitting 2023-07-24 20:11:00,403 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,34917,1690229443609-splitting dir is empty, no logs to split. 2023-07-24 20:11:00,403 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,34917,1690229443609 WAL count=0, meta=true 2023-07-24 20:11:00,405 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,34917,1690229443609-splitting dir is empty, no logs to split. 2023-07-24 20:11:00,405 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,34917,1690229443609 WAL count=0, meta=true 2023-07-24 20:11:00,406 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,34917,1690229443609 WAL splitting is done? wals=0, meta=true 2023-07-24 20:11:00,406 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-24 20:11:00,407 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-24 20:11:00,410 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-24 20:11:00,434 INFO [RS:0;jenkins-hbase4:46705] regionserver.HRegionServer(951): ClusterId : dc54b028-3544-4133-8f0e-d550a19bc790 2023-07-24 20:11:00,434 INFO [RS:2;jenkins-hbase4:46065] regionserver.HRegionServer(951): ClusterId : dc54b028-3544-4133-8f0e-d550a19bc790 2023-07-24 20:11:00,435 DEBUG [RS:0;jenkins-hbase4:46705] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-24 20:11:00,436 DEBUG [RS:2;jenkins-hbase4:46065] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-24 20:11:00,434 INFO [RS:1;jenkins-hbase4:45019] regionserver.HRegionServer(951): ClusterId : dc54b028-3544-4133-8f0e-d550a19bc790 2023-07-24 20:11:00,437 DEBUG [RS:1;jenkins-hbase4:45019] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-24 20:11:00,441 DEBUG [RS:0;jenkins-hbase4:46705] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-24 20:11:00,441 DEBUG [RS:2;jenkins-hbase4:46065] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-24 20:11:00,441 DEBUG [RS:0;jenkins-hbase4:46705] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-24 20:11:00,441 DEBUG [RS:2;jenkins-hbase4:46065] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-24 20:11:00,441 DEBUG [RS:1;jenkins-hbase4:45019] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-24 20:11:00,441 DEBUG [RS:1;jenkins-hbase4:45019] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-24 20:11:00,443 DEBUG [RS:0;jenkins-hbase4:46705] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-24 20:11:00,444 DEBUG [RS:1;jenkins-hbase4:45019] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-24 20:11:00,444 DEBUG [RS:2;jenkins-hbase4:46065] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-24 20:11:00,444 DEBUG [RS:0;jenkins-hbase4:46705] zookeeper.ReadOnlyZKClient(139): Connect 0x104fb07d to 127.0.0.1:62881 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-24 20:11:00,445 DEBUG [RS:1;jenkins-hbase4:45019] zookeeper.ReadOnlyZKClient(139): Connect 0x4b85d299 to 127.0.0.1:62881 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-24 20:11:00,445 DEBUG [RS:2;jenkins-hbase4:46065] zookeeper.ReadOnlyZKClient(139): Connect 0x482a57c8 to 127.0.0.1:62881 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-24 20:11:00,454 DEBUG [RS:0;jenkins-hbase4:46705] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1e885c04, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-24 20:11:00,454 DEBUG [RS:2;jenkins-hbase4:46065] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@46edb0bd, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-24 20:11:00,454 DEBUG [RS:1;jenkins-hbase4:45019] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@45423599, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-24 20:11:00,454 DEBUG [RS:2;jenkins-hbase4:46065] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@42d23cb8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-24 20:11:00,454 DEBUG [RS:0;jenkins-hbase4:46705] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1ab7db0d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-24 20:11:00,454 DEBUG [RS:1;jenkins-hbase4:45019] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@417db9e9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-24 20:11:00,463 DEBUG [RS:1;jenkins-hbase4:45019] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:45019 2023-07-24 20:11:00,464 INFO [RS:1;jenkins-hbase4:45019] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-24 20:11:00,464 INFO [RS:1;jenkins-hbase4:45019] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-24 20:11:00,464 DEBUG [RS:1;jenkins-hbase4:45019] regionserver.HRegionServer(1022): About to register with Master. 2023-07-24 20:11:00,464 INFO [RS:1;jenkins-hbase4:45019] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,33409,1690229459778 with isa=jenkins-hbase4.apache.org/172.31.14.131:45019, startcode=1690229459931 2023-07-24 20:11:00,464 DEBUG [RS:1;jenkins-hbase4:45019] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-24 20:11:00,466 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58401, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.6 (auth:SIMPLE), service=RegionServerStatusService 2023-07-24 20:11:00,467 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=33409] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,45019,1690229459931 2023-07-24 20:11:00,467 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33409,1690229459778] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(787): Updating default servers. 2023-07-24 20:11:00,468 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33409,1690229459778] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(792): Updated with servers: 1 2023-07-24 20:11:00,468 DEBUG [RS:1;jenkins-hbase4:45019] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4 2023-07-24 20:11:00,468 DEBUG [RS:1;jenkins-hbase4:45019] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:37609 2023-07-24 20:11:00,468 DEBUG [RS:1;jenkins-hbase4:45019] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=37241 2023-07-24 20:11:00,469 DEBUG [RS:0;jenkins-hbase4:46705] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:46705 2023-07-24 20:11:00,469 DEBUG [RS:2;jenkins-hbase4:46065] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:46065 2023-07-24 20:11:00,469 INFO [RS:0;jenkins-hbase4:46705] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-24 20:11:00,469 INFO [RS:0;jenkins-hbase4:46705] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-24 20:11:00,469 INFO [RS:2;jenkins-hbase4:46065] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-24 20:11:00,469 INFO [RS:2;jenkins-hbase4:46065] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-24 20:11:00,469 DEBUG [RS:0;jenkins-hbase4:46705] regionserver.HRegionServer(1022): About to register with Master. 2023-07-24 20:11:00,469 DEBUG [RS:2;jenkins-hbase4:46065] regionserver.HRegionServer(1022): About to register with Master. 2023-07-24 20:11:00,473 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:11:00,473 INFO [RS:2;jenkins-hbase4:46065] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,33409,1690229459778 with isa=jenkins-hbase4.apache.org/172.31.14.131:46065, startcode=1690229460063 2023-07-24 20:11:00,473 INFO [RS:0;jenkins-hbase4:46705] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,33409,1690229459778 with isa=jenkins-hbase4.apache.org/172.31.14.131:46705, startcode=1690229459880 2023-07-24 20:11:00,473 DEBUG [RS:1;jenkins-hbase4:45019] zookeeper.ZKUtil(162): regionserver:45019-0x10198f4f3ce0012, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45019,1690229459931 2023-07-24 20:11:00,473 DEBUG [RS:0;jenkins-hbase4:46705] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-24 20:11:00,473 WARN [RS:1;jenkins-hbase4:45019] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-24 20:11:00,473 DEBUG [RS:2;jenkins-hbase4:46065] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-24 20:11:00,473 INFO [RS:1;jenkins-hbase4:45019] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-24 20:11:00,474 DEBUG [RS:1;jenkins-hbase4:45019] regionserver.HRegionServer(1948): logDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,45019,1690229459931 2023-07-24 20:11:00,474 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,45019,1690229459931] 2023-07-24 20:11:00,476 INFO [RS-EventLoopGroup-9-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42149, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.7 (auth:SIMPLE), service=RegionServerStatusService 2023-07-24 20:11:00,476 INFO [RS-EventLoopGroup-9-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42031, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.5 (auth:SIMPLE), service=RegionServerStatusService 2023-07-24 20:11:00,476 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=33409] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,46065,1690229460063 2023-07-24 20:11:00,477 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33409,1690229459778] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(787): Updating default servers. 2023-07-24 20:11:00,477 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33409,1690229459778] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(792): Updated with servers: 2 2023-07-24 20:11:00,477 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=33409] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,46705,1690229459880 2023-07-24 20:11:00,477 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33409,1690229459778] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(787): Updating default servers. 2023-07-24 20:11:00,477 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33409,1690229459778] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(792): Updated with servers: 3 2023-07-24 20:11:00,477 DEBUG [RS:0;jenkins-hbase4:46705] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4 2023-07-24 20:11:00,477 DEBUG [RS:0;jenkins-hbase4:46705] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:37609 2023-07-24 20:11:00,477 DEBUG [RS:0;jenkins-hbase4:46705] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=37241 2023-07-24 20:11:00,479 DEBUG [RS:2;jenkins-hbase4:46065] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4 2023-07-24 20:11:00,479 DEBUG [RS:2;jenkins-hbase4:46065] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:37609 2023-07-24 20:11:00,479 DEBUG [RS:2;jenkins-hbase4:46065] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=37241 2023-07-24 20:11:00,479 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:11:00,479 DEBUG [RS:0;jenkins-hbase4:46705] zookeeper.ZKUtil(162): regionserver:46705-0x10198f4f3ce0011, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46705,1690229459880 2023-07-24 20:11:00,479 WARN [RS:0;jenkins-hbase4:46705] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-24 20:11:00,479 INFO [RS:0;jenkins-hbase4:46705] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-24 20:11:00,479 DEBUG [RS:0;jenkins-hbase4:46705] regionserver.HRegionServer(1948): logDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,46705,1690229459880 2023-07-24 20:11:00,480 DEBUG [RS:2;jenkins-hbase4:46065] zookeeper.ZKUtil(162): regionserver:46065-0x10198f4f3ce0013, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46065,1690229460063 2023-07-24 20:11:00,480 WARN [RS:2;jenkins-hbase4:46065] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-24 20:11:00,480 INFO [RS:2;jenkins-hbase4:46065] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-24 20:11:00,480 DEBUG [RS:2;jenkins-hbase4:46065] regionserver.HRegionServer(1948): logDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,46065,1690229460063 2023-07-24 20:11:00,480 DEBUG [RS:1;jenkins-hbase4:45019] zookeeper.ZKUtil(162): regionserver:45019-0x10198f4f3ce0012, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45019,1690229459931 2023-07-24 20:11:00,482 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,46065,1690229460063] 2023-07-24 20:11:00,482 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,46705,1690229459880] 2023-07-24 20:11:00,484 DEBUG [RS:1;jenkins-hbase4:45019] zookeeper.ZKUtil(162): regionserver:45019-0x10198f4f3ce0012, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46065,1690229460063 2023-07-24 20:11:00,485 DEBUG [RS:1;jenkins-hbase4:45019] zookeeper.ZKUtil(162): regionserver:45019-0x10198f4f3ce0012, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46705,1690229459880 2023-07-24 20:11:00,486 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33409,1690229459778] ipc.AbstractRpcClient(347): Not trying to connect to jenkins-hbase4.apache.org/172.31.14.131:34917 this server is in the failed servers list 2023-07-24 20:11:00,487 DEBUG [RS:2;jenkins-hbase4:46065] zookeeper.ZKUtil(162): regionserver:46065-0x10198f4f3ce0013, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45019,1690229459931 2023-07-24 20:11:00,487 DEBUG [RS:1;jenkins-hbase4:45019] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-24 20:11:00,487 DEBUG [RS:0;jenkins-hbase4:46705] zookeeper.ZKUtil(162): regionserver:46705-0x10198f4f3ce0011, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45019,1690229459931 2023-07-24 20:11:00,487 INFO [RS:1;jenkins-hbase4:45019] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-24 20:11:00,487 DEBUG [RS:2;jenkins-hbase4:46065] zookeeper.ZKUtil(162): regionserver:46065-0x10198f4f3ce0013, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46065,1690229460063 2023-07-24 20:11:00,487 DEBUG [RS:0;jenkins-hbase4:46705] zookeeper.ZKUtil(162): regionserver:46705-0x10198f4f3ce0011, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46065,1690229460063 2023-07-24 20:11:00,488 DEBUG [RS:2;jenkins-hbase4:46065] zookeeper.ZKUtil(162): regionserver:46065-0x10198f4f3ce0013, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46705,1690229459880 2023-07-24 20:11:00,488 DEBUG [RS:0;jenkins-hbase4:46705] zookeeper.ZKUtil(162): regionserver:46705-0x10198f4f3ce0011, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46705,1690229459880 2023-07-24 20:11:00,489 DEBUG [RS:0;jenkins-hbase4:46705] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-24 20:11:00,489 DEBUG [RS:2;jenkins-hbase4:46065] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-24 20:11:00,489 INFO [RS:0;jenkins-hbase4:46705] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-24 20:11:00,489 INFO [RS:2;jenkins-hbase4:46065] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-24 20:11:00,493 INFO [RS:1;jenkins-hbase4:45019] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-24 20:11:00,494 INFO [RS:2;jenkins-hbase4:46065] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-24 20:11:00,495 INFO [RS:0;jenkins-hbase4:46705] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-24 20:11:00,495 INFO [RS:1;jenkins-hbase4:45019] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-24 20:11:00,495 INFO [RS:1;jenkins-hbase4:45019] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:00,495 INFO [RS:2;jenkins-hbase4:46065] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-24 20:11:00,495 INFO [RS:0;jenkins-hbase4:46705] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-24 20:11:00,495 INFO [RS:2;jenkins-hbase4:46065] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:00,495 INFO [RS:0;jenkins-hbase4:46705] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:00,498 INFO [RS:1;jenkins-hbase4:45019] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-24 20:11:00,498 INFO [RS:0;jenkins-hbase4:46705] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-24 20:11:00,499 INFO [RS:2;jenkins-hbase4:46065] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-24 20:11:00,500 INFO [RS:1;jenkins-hbase4:45019] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:00,500 INFO [RS:0;jenkins-hbase4:46705] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:00,500 INFO [RS:2;jenkins-hbase4:46065] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:00,501 DEBUG [RS:1;jenkins-hbase4:45019] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:00,501 DEBUG [RS:2;jenkins-hbase4:46065] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:00,502 DEBUG [RS:1;jenkins-hbase4:45019] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:00,502 DEBUG [RS:2;jenkins-hbase4:46065] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:00,501 DEBUG [RS:0;jenkins-hbase4:46705] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:00,502 DEBUG [RS:2;jenkins-hbase4:46065] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:00,502 DEBUG [RS:0;jenkins-hbase4:46705] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:00,502 DEBUG [RS:2;jenkins-hbase4:46065] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:00,502 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=3; waited=101ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-24 20:11:00,502 DEBUG [RS:2;jenkins-hbase4:46065] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:00,502 DEBUG [RS:1;jenkins-hbase4:45019] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:00,502 DEBUG [RS:2;jenkins-hbase4:46065] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-24 20:11:00,502 DEBUG [RS:1;jenkins-hbase4:45019] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:00,502 DEBUG [RS:2;jenkins-hbase4:46065] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:00,502 DEBUG [RS:1;jenkins-hbase4:45019] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:00,502 DEBUG [RS:0;jenkins-hbase4:46705] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:00,502 DEBUG [RS:1;jenkins-hbase4:45019] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-24 20:11:00,503 DEBUG [RS:0;jenkins-hbase4:46705] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:00,502 DEBUG [RS:2;jenkins-hbase4:46065] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:00,503 DEBUG [RS:0;jenkins-hbase4:46705] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:00,503 DEBUG [RS:2;jenkins-hbase4:46065] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:00,503 DEBUG [RS:0;jenkins-hbase4:46705] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-24 20:11:00,503 DEBUG [RS:1;jenkins-hbase4:45019] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:00,503 DEBUG [RS:0;jenkins-hbase4:46705] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:00,503 DEBUG [RS:1;jenkins-hbase4:45019] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:00,503 DEBUG [RS:2;jenkins-hbase4:46065] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:00,503 DEBUG [RS:1;jenkins-hbase4:45019] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:00,503 DEBUG [RS:0;jenkins-hbase4:46705] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:00,503 DEBUG [RS:1;jenkins-hbase4:45019] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:00,503 DEBUG [RS:0;jenkins-hbase4:46705] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:00,503 DEBUG [RS:0;jenkins-hbase4:46705] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:00,511 INFO [RS:2;jenkins-hbase4:46065] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:00,512 INFO [RS:1;jenkins-hbase4:45019] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:00,512 INFO [RS:0;jenkins-hbase4:46705] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:00,512 INFO [RS:2;jenkins-hbase4:46065] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:00,512 INFO [RS:0;jenkins-hbase4:46705] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:00,512 INFO [RS:2;jenkins-hbase4:46065] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:00,512 INFO [RS:0;jenkins-hbase4:46705] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:00,512 INFO [RS:2;jenkins-hbase4:46065] hbase.ChoreService(166): Chore ScheduledChore name=FileSystemUtilizationChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:00,513 INFO [RS:0;jenkins-hbase4:46705] hbase.ChoreService(166): Chore ScheduledChore name=FileSystemUtilizationChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:00,512 INFO [RS:1;jenkins-hbase4:45019] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:00,513 INFO [RS:1;jenkins-hbase4:45019] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:00,513 INFO [RS:1;jenkins-hbase4:45019] hbase.ChoreService(166): Chore ScheduledChore name=FileSystemUtilizationChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:00,523 INFO [RS:1;jenkins-hbase4:45019] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-24 20:11:00,523 INFO [RS:0;jenkins-hbase4:46705] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-24 20:11:00,523 INFO [RS:1;jenkins-hbase4:45019] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45019,1690229459931-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:00,523 INFO [RS:0;jenkins-hbase4:46705] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46705,1690229459880-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:00,523 INFO [RS:2;jenkins-hbase4:46065] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-24 20:11:00,523 INFO [RS:2;jenkins-hbase4:46065] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46065,1690229460063-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:00,533 INFO [RS:1;jenkins-hbase4:45019] regionserver.Replication(203): jenkins-hbase4.apache.org,45019,1690229459931 started 2023-07-24 20:11:00,533 INFO [RS:1;jenkins-hbase4:45019] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,45019,1690229459931, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:45019, sessionid=0x10198f4f3ce0012 2023-07-24 20:11:00,534 DEBUG [RS:1;jenkins-hbase4:45019] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-24 20:11:00,534 DEBUG [RS:1;jenkins-hbase4:45019] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,45019,1690229459931 2023-07-24 20:11:00,534 DEBUG [RS:1;jenkins-hbase4:45019] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,45019,1690229459931' 2023-07-24 20:11:00,534 INFO [RS:2;jenkins-hbase4:46065] regionserver.Replication(203): jenkins-hbase4.apache.org,46065,1690229460063 started 2023-07-24 20:11:00,534 DEBUG [RS:1;jenkins-hbase4:45019] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 20:11:00,534 INFO [RS:2;jenkins-hbase4:46065] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,46065,1690229460063, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:46065, sessionid=0x10198f4f3ce0013 2023-07-24 20:11:00,534 INFO [RS:0;jenkins-hbase4:46705] regionserver.Replication(203): jenkins-hbase4.apache.org,46705,1690229459880 started 2023-07-24 20:11:00,534 DEBUG [RS:2;jenkins-hbase4:46065] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-24 20:11:00,534 DEBUG [RS:2;jenkins-hbase4:46065] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,46065,1690229460063 2023-07-24 20:11:00,534 DEBUG [RS:2;jenkins-hbase4:46065] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46065,1690229460063' 2023-07-24 20:11:00,534 DEBUG [RS:2;jenkins-hbase4:46065] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 20:11:00,534 INFO [RS:0;jenkins-hbase4:46705] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,46705,1690229459880, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:46705, sessionid=0x10198f4f3ce0011 2023-07-24 20:11:00,534 DEBUG [RS:0;jenkins-hbase4:46705] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-24 20:11:00,534 DEBUG [RS:1;jenkins-hbase4:45019] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 20:11:00,534 DEBUG [RS:0;jenkins-hbase4:46705] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,46705,1690229459880 2023-07-24 20:11:00,534 DEBUG [RS:0;jenkins-hbase4:46705] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46705,1690229459880' 2023-07-24 20:11:00,534 DEBUG [RS:0;jenkins-hbase4:46705] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 20:11:00,534 DEBUG [RS:2;jenkins-hbase4:46065] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 20:11:00,535 DEBUG [RS:1;jenkins-hbase4:45019] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-24 20:11:00,535 DEBUG [RS:1;jenkins-hbase4:45019] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-24 20:11:00,535 DEBUG [RS:1;jenkins-hbase4:45019] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,45019,1690229459931 2023-07-24 20:11:00,535 DEBUG [RS:1;jenkins-hbase4:45019] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,45019,1690229459931' 2023-07-24 20:11:00,535 DEBUG [RS:1;jenkins-hbase4:45019] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-24 20:11:00,535 DEBUG [RS:2;jenkins-hbase4:46065] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-24 20:11:00,535 DEBUG [RS:2;jenkins-hbase4:46065] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-24 20:11:00,535 DEBUG [RS:2;jenkins-hbase4:46065] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,46065,1690229460063 2023-07-24 20:11:00,535 DEBUG [RS:2;jenkins-hbase4:46065] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46065,1690229460063' 2023-07-24 20:11:00,535 DEBUG [RS:2;jenkins-hbase4:46065] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-24 20:11:00,535 DEBUG [RS:0;jenkins-hbase4:46705] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 20:11:00,535 DEBUG [RS:1;jenkins-hbase4:45019] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-24 20:11:00,535 DEBUG [RS:2;jenkins-hbase4:46065] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-24 20:11:00,535 DEBUG [RS:0;jenkins-hbase4:46705] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-24 20:11:00,535 DEBUG [RS:0;jenkins-hbase4:46705] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-24 20:11:00,535 DEBUG [RS:0;jenkins-hbase4:46705] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,46705,1690229459880 2023-07-24 20:11:00,535 DEBUG [RS:0;jenkins-hbase4:46705] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46705,1690229459880' 2023-07-24 20:11:00,535 DEBUG [RS:1;jenkins-hbase4:45019] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-24 20:11:00,535 DEBUG [RS:0;jenkins-hbase4:46705] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-24 20:11:00,536 INFO [RS:1;jenkins-hbase4:45019] quotas.RegionServerRpcQuotaManager(67): Initializing RPC quota support 2023-07-24 20:11:00,536 DEBUG [RS:2;jenkins-hbase4:46065] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-24 20:11:00,536 INFO [RS:2;jenkins-hbase4:46065] quotas.RegionServerRpcQuotaManager(67): Initializing RPC quota support 2023-07-24 20:11:00,536 DEBUG [RS:0;jenkins-hbase4:46705] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-24 20:11:00,536 DEBUG [RS:0;jenkins-hbase4:46705] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-24 20:11:00,536 INFO [RS:0;jenkins-hbase4:46705] quotas.RegionServerRpcQuotaManager(67): Initializing RPC quota support 2023-07-24 20:11:00,538 INFO [RS:2;jenkins-hbase4:46065] hbase.ChoreService(166): Chore ScheduledChore name=QuotaRefresherChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:00,538 INFO [RS:0;jenkins-hbase4:46705] hbase.ChoreService(166): Chore ScheduledChore name=QuotaRefresherChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:00,538 INFO [RS:1;jenkins-hbase4:45019] hbase.ChoreService(166): Chore ScheduledChore name=QuotaRefresherChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:00,538 DEBUG [RS:2;jenkins-hbase4:46065] zookeeper.ZKUtil(398): regionserver:46065-0x10198f4f3ce0013, quorum=127.0.0.1:62881, baseZNode=/hbase Unable to get data of znode /hbase/rpc-throttle because node does not exist (not an error) 2023-07-24 20:11:00,539 DEBUG [RS:1;jenkins-hbase4:45019] zookeeper.ZKUtil(398): regionserver:45019-0x10198f4f3ce0012, quorum=127.0.0.1:62881, baseZNode=/hbase Unable to get data of znode /hbase/rpc-throttle because node does not exist (not an error) 2023-07-24 20:11:00,539 INFO [RS:2;jenkins-hbase4:46065] quotas.RegionServerRpcQuotaManager(73): Start rpc quota manager and rpc throttle enabled is true 2023-07-24 20:11:00,539 DEBUG [RS:0;jenkins-hbase4:46705] zookeeper.ZKUtil(398): regionserver:46705-0x10198f4f3ce0011, quorum=127.0.0.1:62881, baseZNode=/hbase Unable to get data of znode /hbase/rpc-throttle because node does not exist (not an error) 2023-07-24 20:11:00,539 INFO [RS:1;jenkins-hbase4:45019] quotas.RegionServerRpcQuotaManager(73): Start rpc quota manager and rpc throttle enabled is true 2023-07-24 20:11:00,539 INFO [RS:0;jenkins-hbase4:46705] quotas.RegionServerRpcQuotaManager(73): Start rpc quota manager and rpc throttle enabled is true 2023-07-24 20:11:00,539 INFO [RS:1;jenkins-hbase4:45019] hbase.ChoreService(166): Chore ScheduledChore name=SpaceQuotaRefresherChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:00,539 INFO [RS:2;jenkins-hbase4:46065] hbase.ChoreService(166): Chore ScheduledChore name=SpaceQuotaRefresherChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:00,539 INFO [RS:0;jenkins-hbase4:46705] hbase.ChoreService(166): Chore ScheduledChore name=SpaceQuotaRefresherChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:00,539 INFO [RS:0;jenkins-hbase4:46705] hbase.ChoreService(166): Chore ScheduledChore name=RegionSizeReportingChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:00,539 INFO [RS:2;jenkins-hbase4:46065] hbase.ChoreService(166): Chore ScheduledChore name=RegionSizeReportingChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:00,539 INFO [RS:1;jenkins-hbase4:45019] hbase.ChoreService(166): Chore ScheduledChore name=RegionSizeReportingChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:00,561 DEBUG [jenkins-hbase4:33409] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-07-24 20:11:00,561 DEBUG [jenkins-hbase4:33409] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-24 20:11:00,561 DEBUG [jenkins-hbase4:33409] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 20:11:00,561 DEBUG [jenkins-hbase4:33409] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 20:11:00,561 DEBUG [jenkins-hbase4:33409] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 20:11:00,561 DEBUG [jenkins-hbase4:33409] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 20:11:00,563 INFO [PEWorker-2] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,46705,1690229459880, state=OPENING 2023-07-24 20:11:00,566 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-07-24 20:11:00,566 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-24 20:11:00,566 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,46705,1690229459880}] 2023-07-24 20:11:00,643 INFO [RS:1;jenkins-hbase4:45019] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45019%2C1690229459931, suffix=, logDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,45019,1690229459931, archiveDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/oldWALs, maxLogs=32 2023-07-24 20:11:00,643 INFO [RS:0;jenkins-hbase4:46705] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46705%2C1690229459880, suffix=, logDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,46705,1690229459880, archiveDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/oldWALs, maxLogs=32 2023-07-24 20:11:00,643 INFO [RS:2;jenkins-hbase4:46065] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46065%2C1690229460063, suffix=, logDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,46065,1690229460063, archiveDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/oldWALs, maxLogs=32 2023-07-24 20:11:00,663 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK] 2023-07-24 20:11:00,663 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK] 2023-07-24 20:11:00,663 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK] 2023-07-24 20:11:00,669 INFO [RS:1;jenkins-hbase4:45019] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,45019,1690229459931/jenkins-hbase4.apache.org%2C45019%2C1690229459931.1690229460645 2023-07-24 20:11:00,669 DEBUG [RS:1;jenkins-hbase4:45019] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK], DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK], DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK]] 2023-07-24 20:11:00,673 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK] 2023-07-24 20:11:00,673 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK] 2023-07-24 20:11:00,673 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK] 2023-07-24 20:11:00,675 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK] 2023-07-24 20:11:00,676 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK] 2023-07-24 20:11:00,676 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK] 2023-07-24 20:11:00,678 INFO [RS:0;jenkins-hbase4:46705] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,46705,1690229459880/jenkins-hbase4.apache.org%2C46705%2C1690229459880.1690229460652 2023-07-24 20:11:00,679 DEBUG [RS:0;jenkins-hbase4:46705] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK], DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK], DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK]] 2023-07-24 20:11:00,680 INFO [RS:2;jenkins-hbase4:46065] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,46065,1690229460063/jenkins-hbase4.apache.org%2C46065%2C1690229460063.1690229460652 2023-07-24 20:11:00,680 DEBUG [RS:2;jenkins-hbase4:46065] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK], DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK], DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK]] 2023-07-24 20:11:00,687 WARN [ReadOnlyZKClient-127.0.0.1:62881@0x4d670eec] client.ZKConnectionRegistry(168): Meta region is in state OPENING 2023-07-24 20:11:00,687 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33409,1690229459778] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-24 20:11:00,689 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43016, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-24 20:11:00,689 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46705] ipc.CallRunner(144): callId: 2 service: ClientService methodName: Get size: 88 connection: 172.31.14.131:43016 deadline: 1690229520689, exception=org.apache.hadoop.hbase.NotServingRegionException: hbase:meta,,1 is not online on jenkins-hbase4.apache.org,46705,1690229459880 2023-07-24 20:11:00,720 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,46705,1690229459880 2023-07-24 20:11:00,722 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-24 20:11:00,723 INFO [RS-EventLoopGroup-10-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43018, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-24 20:11:00,729 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-24 20:11:00,729 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-24 20:11:00,732 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46705%2C1690229459880.meta, suffix=.meta, logDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,46705,1690229459880, archiveDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/oldWALs, maxLogs=32 2023-07-24 20:11:00,749 DEBUG [RS-EventLoopGroup-12-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK] 2023-07-24 20:11:00,749 DEBUG [RS-EventLoopGroup-12-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK] 2023-07-24 20:11:00,749 DEBUG [RS-EventLoopGroup-12-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK] 2023-07-24 20:11:00,751 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,46705,1690229459880/jenkins-hbase4.apache.org%2C46705%2C1690229459880.meta.1690229460732.meta 2023-07-24 20:11:00,751 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK], DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK], DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK]] 2023-07-24 20:11:00,752 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-24 20:11:00,752 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-24 20:11:00,752 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-24 20:11:00,752 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-24 20:11:00,752 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-24 20:11:00,752 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:11:00,752 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-24 20:11:00,752 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-24 20:11:00,754 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-24 20:11:00,755 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/info 2023-07-24 20:11:00,755 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/info 2023-07-24 20:11:00,756 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-24 20:11:00,768 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/info/14fae0676b7b4b9dbef6a9c00b67e86e 2023-07-24 20:11:00,774 INFO [StoreFileOpener-info-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 71cc04e9686e4212a8a52f78335cc5c6 2023-07-24 20:11:00,775 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/info/71cc04e9686e4212a8a52f78335cc5c6 2023-07-24 20:11:00,775 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:11:00,775 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-24 20:11:00,776 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/rep_barrier 2023-07-24 20:11:00,776 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/rep_barrier 2023-07-24 20:11:00,776 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-24 20:11:00,784 INFO [StoreFileOpener-rep_barrier-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d4449076caff44088d8e3a03756fcbd6 2023-07-24 20:11:00,784 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/rep_barrier/d4449076caff44088d8e3a03756fcbd6 2023-07-24 20:11:00,784 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:11:00,784 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-24 20:11:00,785 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/table 2023-07-24 20:11:00,785 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/table 2023-07-24 20:11:00,785 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-24 20:11:00,791 INFO [StoreFileOpener-table-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 92a06725a50449fa925e17781e43f1e3 2023-07-24 20:11:00,791 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/table/92a06725a50449fa925e17781e43f1e3 2023-07-24 20:11:00,795 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/table/a90c3b62969d4417a8b965913cf8fbbc 2023-07-24 20:11:00,796 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:11:00,796 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740 2023-07-24 20:11:00,797 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740 2023-07-24 20:11:00,799 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-24 20:11:00,800 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-24 20:11:00,801 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=144; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=11048349120, jitterRate=0.02895769476890564}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-24 20:11:00,801 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-24 20:11:00,801 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=114, masterSystemTime=1690229460720 2023-07-24 20:11:00,805 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-24 20:11:00,806 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-24 20:11:00,806 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,46705,1690229459880, state=OPEN 2023-07-24 20:11:00,808 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-07-24 20:11:00,808 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-24 20:11:00,810 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-07-24 20:11:00,810 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,46705,1690229459880 in 242 msec 2023-07-24 20:11:00,811 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-07-24 20:11:00,811 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 404 msec 2023-07-24 20:11:01,005 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33409,1690229459778] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-24 20:11:01,006 WARN [RS-EventLoopGroup-12-3] ipc.NettyRpcConnection$2(294): Exception encountered while connecting to the server jenkins-hbase4.apache.org/172.31.14.131:39791 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:39791 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:155) at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:128) at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-07-24 20:11:01,006 DEBUG [RS-EventLoopGroup-12-3] ipc.FailedServers(52): Added failed server with address jenkins-hbase4.apache.org/172.31.14.131:39791 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:39791 2023-07-24 20:11:01,117 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33409,1690229459778] ipc.AbstractRpcClient(347): Not trying to connect to jenkins-hbase4.apache.org/172.31.14.131:39791 this server is in the failed servers list 2023-07-24 20:11:01,321 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33409,1690229459778] ipc.AbstractRpcClient(347): Not trying to connect to jenkins-hbase4.apache.org/172.31.14.131:39791 this server is in the failed servers list 2023-07-24 20:11:01,627 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33409,1690229459778] ipc.AbstractRpcClient(347): Not trying to connect to jenkins-hbase4.apache.org/172.31.14.131:39791 this server is in the failed servers list 2023-07-24 20:11:02,005 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=3; waited=1604ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1503ms 2023-07-24 20:11:02,134 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33409,1690229459778] ipc.AbstractRpcClient(347): Not trying to connect to jenkins-hbase4.apache.org/172.31.14.131:39791 this server is in the failed servers list 2023-07-24 20:11:03,146 WARN [RS-EventLoopGroup-12-3] ipc.NettyRpcConnection$2(294): Exception encountered while connecting to the server jenkins-hbase4.apache.org/172.31.14.131:39791 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:39791 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:155) at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:128) at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-07-24 20:11:03,148 DEBUG [RS-EventLoopGroup-12-3] ipc.FailedServers(52): Added failed server with address jenkins-hbase4.apache.org/172.31.14.131:39791 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:39791 2023-07-24 20:11:03,507 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=3; waited=3106ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3005ms 2023-07-24 20:11:04,910 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=4509ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-07-24 20:11:04,910 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 20:11:04,920 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=324410cf2a29c2dca1a19e8570799d47, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39791,1690229440084, regionLocation=jenkins-hbase4.apache.org,39791,1690229440084, openSeqNum=2 2023-07-24 20:11:04,920 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=ec4f892f95beff2c1aa7863da43a18fa, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,39791,1690229440084, regionLocation=jenkins-hbase4.apache.org,39791,1690229440084, openSeqNum=2 2023-07-24 20:11:04,920 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 20:11:04,920 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690229524920 2023-07-24 20:11:04,920 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690229584920 2023-07-24 20:11:04,920 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 10 msec 2023-07-24 20:11:04,940 INFO [PEWorker-2] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,34917,1690229443609 had 1 regions 2023-07-24 20:11:04,940 INFO [PEWorker-4] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,34483,1690229439858 had 0 regions 2023-07-24 20:11:04,940 INFO [PEWorker-3] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,45209,1690229445078 had 0 regions 2023-07-24 20:11:04,940 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33409,1690229459778-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:04,940 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33409,1690229459778-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:04,942 INFO [PEWorker-1] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,39791,1690229440084 had 2 regions 2023-07-24 20:11:04,942 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33409,1690229459778-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:04,942 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:33409, period=300000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:04,942 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:04,942 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. is NOT online; state={324410cf2a29c2dca1a19e8570799d47 state=OPEN, ts=1690229464920, server=jenkins-hbase4.apache.org,39791,1690229440084}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-07-24 20:11:04,943 INFO [PEWorker-3] procedure.ServerCrashProcedure(300): Splitting WALs pid=110, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,45209,1690229445078, splitWal=true, meta=false, isMeta: false 2023-07-24 20:11:04,947 INFO [PEWorker-2] procedure.ServerCrashProcedure(300): Splitting WALs pid=112, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,34917,1690229443609, splitWal=true, meta=true, isMeta: false 2023-07-24 20:11:04,948 INFO [PEWorker-4] procedure.ServerCrashProcedure(300): Splitting WALs pid=111, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,34483,1690229439858, splitWal=true, meta=false, isMeta: false 2023-07-24 20:11:04,948 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=109, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,39791,1690229440084, splitWal=true, meta=false, isMeta: false 2023-07-24 20:11:04,949 DEBUG [PEWorker-3] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,45209,1690229445078-splitting 2023-07-24 20:11:04,951 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,45209,1690229445078-splitting dir is empty, no logs to split. 2023-07-24 20:11:04,951 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,45209,1690229445078 WAL count=0, meta=false 2023-07-24 20:11:04,952 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,34917,1690229443609-splitting dir is empty, no logs to split. 2023-07-24 20:11:04,952 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase4.apache.org,34917,1690229443609 WAL count=0, meta=false 2023-07-24 20:11:04,956 DEBUG [PEWorker-4] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,34483,1690229439858-splitting 2023-07-24 20:11:04,957 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,34483,1690229439858-splitting dir is empty, no logs to split. 2023-07-24 20:11:04,957 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,34483,1690229439858 WAL count=0, meta=false 2023-07-24 20:11:04,958 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,39791,1690229440084-splitting 2023-07-24 20:11:04,959 WARN [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase4.apache.org,39791,1690229440084/hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47., unknown_server=jenkins-hbase4.apache.org,39791,1690229440084/hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. 2023-07-24 20:11:04,960 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,39791,1690229440084-splitting dir is empty, no logs to split. 2023-07-24 20:11:04,960 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,39791,1690229440084 WAL count=0, meta=false 2023-07-24 20:11:04,961 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,45209,1690229445078-splitting dir is empty, no logs to split. 2023-07-24 20:11:04,961 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,45209,1690229445078 WAL count=0, meta=false 2023-07-24 20:11:04,961 DEBUG [PEWorker-3] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,45209,1690229445078 WAL splitting is done? wals=0, meta=false 2023-07-24 20:11:04,962 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,34917,1690229443609-splitting dir is empty, no logs to split. 2023-07-24 20:11:04,962 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase4.apache.org,34917,1690229443609 WAL count=0, meta=false 2023-07-24 20:11:04,962 DEBUG [PEWorker-2] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,34917,1690229443609 WAL splitting is done? wals=0, meta=false 2023-07-24 20:11:04,963 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,34483,1690229439858-splitting dir is empty, no logs to split. 2023-07-24 20:11:04,963 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,34483,1690229439858 WAL count=0, meta=false 2023-07-24 20:11:04,963 DEBUG [PEWorker-4] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,34483,1690229439858 WAL splitting is done? wals=0, meta=false 2023-07-24 20:11:04,964 INFO [PEWorker-2] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,34917,1690229443609 after splitting done 2023-07-24 20:11:04,964 DEBUG [PEWorker-2] master.DeadServer(114): Removed jenkins-hbase4.apache.org,34917,1690229443609 from processing; numProcessing=3 2023-07-24 20:11:04,964 INFO [PEWorker-3] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,45209,1690229445078 failed, ignore...File hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,45209,1690229445078-splitting does not exist. 2023-07-24 20:11:04,965 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,39791,1690229440084-splitting dir is empty, no logs to split. 2023-07-24 20:11:04,965 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,39791,1690229440084 WAL count=0, meta=false 2023-07-24 20:11:04,965 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,39791,1690229440084 WAL splitting is done? wals=0, meta=false 2023-07-24 20:11:04,966 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,34917,1690229443609, splitWal=true, meta=true in 4.6090 sec 2023-07-24 20:11:04,966 INFO [PEWorker-3] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,45209,1690229445078 after splitting done 2023-07-24 20:11:04,966 DEBUG [PEWorker-3] master.DeadServer(114): Removed jenkins-hbase4.apache.org,45209,1690229445078 from processing; numProcessing=2 2023-07-24 20:11:04,966 INFO [PEWorker-4] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,34483,1690229439858 failed, ignore...File hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,34483,1690229439858-splitting does not exist. 2023-07-24 20:11:04,967 INFO [PEWorker-4] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,34483,1690229439858 after splitting done 2023-07-24 20:11:04,967 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=110, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,45209,1690229445078, splitWal=true, meta=false in 4.6150 sec 2023-07-24 20:11:04,967 DEBUG [PEWorker-4] master.DeadServer(114): Removed jenkins-hbase4.apache.org,34483,1690229439858 from processing; numProcessing=1 2023-07-24 20:11:04,968 INFO [PEWorker-1] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,39791,1690229440084 failed, ignore...File hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,39791,1690229440084-splitting does not exist. 2023-07-24 20:11:04,969 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=111, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,34483,1690229439858, splitWal=true, meta=false in 4.6140 sec 2023-07-24 20:11:04,969 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:rsgroup, region=ec4f892f95beff2c1aa7863da43a18fa, ASSIGN}, {pid=116, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=324410cf2a29c2dca1a19e8570799d47, ASSIGN}] 2023-07-24 20:11:04,969 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:rsgroup, region=ec4f892f95beff2c1aa7863da43a18fa, ASSIGN 2023-07-24 20:11:04,969 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=324410cf2a29c2dca1a19e8570799d47, ASSIGN 2023-07-24 20:11:04,970 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:rsgroup, region=ec4f892f95beff2c1aa7863da43a18fa, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-24 20:11:04,970 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=324410cf2a29c2dca1a19e8570799d47, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-24 20:11:04,970 DEBUG [jenkins-hbase4:33409] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-07-24 20:11:04,971 DEBUG [jenkins-hbase4:33409] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-24 20:11:04,971 DEBUG [jenkins-hbase4:33409] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 20:11:04,971 DEBUG [jenkins-hbase4:33409] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 20:11:04,971 DEBUG [jenkins-hbase4:33409] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 20:11:04,971 DEBUG [jenkins-hbase4:33409] balancer.BaseLoadBalancer$Cluster(378): Number of tables=2, number of hosts=1, number of racks=1 2023-07-24 20:11:04,973 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=ec4f892f95beff2c1aa7863da43a18fa, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46705,1690229459880 2023-07-24 20:11:04,973 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=324410cf2a29c2dca1a19e8570799d47, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46065,1690229460063 2023-07-24 20:11:04,973 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa.","families":{"info":[{"qualifier":"regioninfo","vlen":39,"tag":[],"timestamp":"1690229464973"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229464973"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229464973"}]},"ts":"1690229464973"} 2023-07-24 20:11:04,973 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690229464973"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229464973"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229464973"}]},"ts":"1690229464973"} 2023-07-24 20:11:04,975 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=115, state=RUNNABLE; OpenRegionProcedure ec4f892f95beff2c1aa7863da43a18fa, server=jenkins-hbase4.apache.org,46705,1690229459880}] 2023-07-24 20:11:04,975 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=116, state=RUNNABLE; OpenRegionProcedure 324410cf2a29c2dca1a19e8570799d47, server=jenkins-hbase4.apache.org,46065,1690229460063}] 2023-07-24 20:11:05,128 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,46065,1690229460063 2023-07-24 20:11:05,129 DEBUG [RSProcedureDispatcher-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-24 20:11:05,130 INFO [RS-EventLoopGroup-12-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55942, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-24 20:11:05,135 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. 2023-07-24 20:11:05,135 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 324410cf2a29c2dca1a19e8570799d47, NAME => 'hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47.', STARTKEY => '', ENDKEY => ''} 2023-07-24 20:11:05,135 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. 2023-07-24 20:11:05,135 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ec4f892f95beff2c1aa7863da43a18fa, NAME => 'hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa.', STARTKEY => '', ENDKEY => ''} 2023-07-24 20:11:05,135 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 324410cf2a29c2dca1a19e8570799d47 2023-07-24 20:11:05,135 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:11:05,135 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-24 20:11:05,135 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 324410cf2a29c2dca1a19e8570799d47 2023-07-24 20:11:05,135 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 324410cf2a29c2dca1a19e8570799d47 2023-07-24 20:11:05,135 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. service=MultiRowMutationService 2023-07-24 20:11:05,136 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:rsgroup successfully. 2023-07-24 20:11:05,136 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table rsgroup ec4f892f95beff2c1aa7863da43a18fa 2023-07-24 20:11:05,136 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:11:05,136 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ec4f892f95beff2c1aa7863da43a18fa 2023-07-24 20:11:05,136 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ec4f892f95beff2c1aa7863da43a18fa 2023-07-24 20:11:05,137 INFO [StoreOpener-ec4f892f95beff2c1aa7863da43a18fa-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family m of region ec4f892f95beff2c1aa7863da43a18fa 2023-07-24 20:11:05,137 INFO [StoreOpener-324410cf2a29c2dca1a19e8570799d47-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 324410cf2a29c2dca1a19e8570799d47 2023-07-24 20:11:05,138 DEBUG [StoreOpener-ec4f892f95beff2c1aa7863da43a18fa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/rsgroup/ec4f892f95beff2c1aa7863da43a18fa/m 2023-07-24 20:11:05,138 DEBUG [StoreOpener-ec4f892f95beff2c1aa7863da43a18fa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/rsgroup/ec4f892f95beff2c1aa7863da43a18fa/m 2023-07-24 20:11:05,139 INFO [StoreOpener-ec4f892f95beff2c1aa7863da43a18fa-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ec4f892f95beff2c1aa7863da43a18fa columnFamilyName m 2023-07-24 20:11:05,144 DEBUG [StoreOpener-324410cf2a29c2dca1a19e8570799d47-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/namespace/324410cf2a29c2dca1a19e8570799d47/info 2023-07-24 20:11:05,144 DEBUG [StoreOpener-324410cf2a29c2dca1a19e8570799d47-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/namespace/324410cf2a29c2dca1a19e8570799d47/info 2023-07-24 20:11:05,145 INFO [StoreOpener-324410cf2a29c2dca1a19e8570799d47-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 324410cf2a29c2dca1a19e8570799d47 columnFamilyName info 2023-07-24 20:11:05,149 INFO [StoreFileOpener-m-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 327d57459138467dbba127685af878e9 2023-07-24 20:11:05,149 DEBUG [StoreOpener-ec4f892f95beff2c1aa7863da43a18fa-1] regionserver.HStore(539): loaded hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/rsgroup/ec4f892f95beff2c1aa7863da43a18fa/m/327d57459138467dbba127685af878e9 2023-07-24 20:11:05,150 INFO [StoreOpener-ec4f892f95beff2c1aa7863da43a18fa-1] regionserver.HStore(310): Store=ec4f892f95beff2c1aa7863da43a18fa/m, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:11:05,151 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/rsgroup/ec4f892f95beff2c1aa7863da43a18fa 2023-07-24 20:11:05,154 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/rsgroup/ec4f892f95beff2c1aa7863da43a18fa 2023-07-24 20:11:05,156 INFO [StoreFileOpener-info-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 81b8365c87814c2e9ac1bfab2b6bb254 2023-07-24 20:11:05,156 DEBUG [StoreOpener-324410cf2a29c2dca1a19e8570799d47-1] regionserver.HStore(539): loaded hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/namespace/324410cf2a29c2dca1a19e8570799d47/info/81b8365c87814c2e9ac1bfab2b6bb254 2023-07-24 20:11:05,156 INFO [StoreOpener-324410cf2a29c2dca1a19e8570799d47-1] regionserver.HStore(310): Store=324410cf2a29c2dca1a19e8570799d47/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:11:05,157 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/namespace/324410cf2a29c2dca1a19e8570799d47 2023-07-24 20:11:05,158 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/namespace/324410cf2a29c2dca1a19e8570799d47 2023-07-24 20:11:05,163 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ec4f892f95beff2c1aa7863da43a18fa 2023-07-24 20:11:05,164 WARN [RS-EventLoopGroup-12-3] ipc.NettyRpcConnection$2(294): Exception encountered while connecting to the server jenkins-hbase4.apache.org/172.31.14.131:39791 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:39791 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:155) at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:128) at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-07-24 20:11:05,164 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ec4f892f95beff2c1aa7863da43a18fa; next sequenceid=71; org.apache.hadoop.hbase.regionserver.DisabledRegionSplitPolicy@13d57ff, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 20:11:05,164 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ec4f892f95beff2c1aa7863da43a18fa: 2023-07-24 20:11:05,166 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33409,1690229459778] client.RpcRetryingCallerImpl(129): Call exception, tries=6, retries=46, started=4169 ms ago, cancelled=false, msg=Call to address=jenkins-hbase4.apache.org/172.31.14.131:39791 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:39791, details=row '\x00' on table 'hbase:rsgroup' at region=hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa., hostname=jenkins-hbase4.apache.org,39791,1690229440084, seqNum=2, see https://s.apache.org/timeout, exception=java.net.ConnectException: Call to address=jenkins-hbase4.apache.org/172.31.14.131:39791 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:39791 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:186) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:385) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:99) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:195) at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$300(NettyRpcConnection.java:76) at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:296) at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:287) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:39791 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:155) at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:128) at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-07-24 20:11:05,167 DEBUG [RS-EventLoopGroup-12-3] ipc.FailedServers(52): Added failed server with address jenkins-hbase4.apache.org/172.31.14.131:39791 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:39791 2023-07-24 20:11:05,169 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa., pid=117, masterSystemTime=1690229465128 2023-07-24 20:11:05,171 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. 2023-07-24 20:11:05,171 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. 2023-07-24 20:11:05,171 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=ec4f892f95beff2c1aa7863da43a18fa, regionState=OPEN, openSeqNum=71, regionLocation=jenkins-hbase4.apache.org,46705,1690229459880 2023-07-24 20:11:05,172 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa.","families":{"info":[{"qualifier":"regioninfo","vlen":39,"tag":[],"timestamp":"1690229465171"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690229465171"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690229465171"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690229465171"}]},"ts":"1690229465171"} 2023-07-24 20:11:05,173 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 324410cf2a29c2dca1a19e8570799d47 2023-07-24 20:11:05,174 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 324410cf2a29c2dca1a19e8570799d47; next sequenceid=15; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=10232761760, jitterRate=-0.046999797224998474}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 20:11:05,174 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 324410cf2a29c2dca1a19e8570799d47: 2023-07-24 20:11:05,175 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47., pid=118, masterSystemTime=1690229465128 2023-07-24 20:11:05,179 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. 2023-07-24 20:11:05,180 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. 2023-07-24 20:11:05,181 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=115 2023-07-24 20:11:05,180 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=324410cf2a29c2dca1a19e8570799d47, regionState=OPEN, openSeqNum=15, regionLocation=jenkins-hbase4.apache.org,46065,1690229460063 2023-07-24 20:11:05,181 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690229465180"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690229465180"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690229465180"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690229465180"}]},"ts":"1690229465180"} 2023-07-24 20:11:05,181 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure ec4f892f95beff2c1aa7863da43a18fa, server=jenkins-hbase4.apache.org,46705,1690229459880 in 203 msec 2023-07-24 20:11:05,183 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=hbase:rsgroup, region=ec4f892f95beff2c1aa7863da43a18fa, ASSIGN in 212 msec 2023-07-24 20:11:05,188 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=116 2023-07-24 20:11:05,188 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 324410cf2a29c2dca1a19e8570799d47, server=jenkins-hbase4.apache.org,46065,1690229460063 in 211 msec 2023-07-24 20:11:05,191 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=116, resume processing ppid=109 2023-07-24 20:11:05,191 INFO [PEWorker-5] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,39791,1690229440084 after splitting done 2023-07-24 20:11:05,191 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=324410cf2a29c2dca1a19e8570799d47, ASSIGN in 219 msec 2023-07-24 20:11:05,191 DEBUG [PEWorker-5] master.DeadServer(114): Removed jenkins-hbase4.apache.org,39791,1690229440084 from processing; numProcessing=0 2023-07-24 20:11:05,192 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,39791,1690229440084, splitWal=true, meta=false in 4.8420 sec 2023-07-24 20:11:05,943 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/namespace 2023-07-24 20:11:05,949 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-24 20:11:05,951 INFO [RS-EventLoopGroup-12-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55956, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-24 20:11:05,963 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-07-24 20:11:05,965 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-07-24 20:11:05,965 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.817sec 2023-07-24 20:11:05,966 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(103): Quota table not found. Creating... 2023-07-24 20:11:05,966 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:quota', {NAME => 'q', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'u', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-24 20:11:05,967 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:quota 2023-07-24 20:11:05,967 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(107): Initializing quota support 2023-07-24 20:11:05,971 INFO [master/jenkins-hbase4:0:becomeActiveMaster] namespace.NamespaceStateManager(59): Namespace State Manager started. 2023-07-24 20:11:05,972 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=119, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:quota execute state=CREATE_TABLE_PRE_OPERATION 2023-07-24 20:11:05,973 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=119, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:quota execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-24 20:11:05,975 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/hbase/quota/4fa2a21e5a49480a149a055b0fc39515 2023-07-24 20:11:05,976 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/hbase/quota/4fa2a21e5a49480a149a055b0fc39515 empty. 2023-07-24 20:11:05,976 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/hbase/quota/4fa2a21e5a49480a149a055b0fc39515 2023-07-24 20:11:05,976 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived hbase:quota regions 2023-07-24 20:11:05,999 INFO [master/jenkins-hbase4:0:becomeActiveMaster] namespace.NamespaceStateManager(222): Finished updating state of 2 namespaces. 2023-07-24 20:11:05,999 INFO [master/jenkins-hbase4:0:becomeActiveMaster] namespace.NamespaceAuditor(50): NamespaceAuditor started. 2023-07-24 20:11:06,002 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=QuotaObserverChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:06,002 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=QuotaObserverChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:06,002 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-24 20:11:06,002 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-24 20:11:06,003 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33409,1690229459778-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-24 20:11:06,003 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33409,1690229459778-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-24 20:11:06,004 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-24 20:11:06,009 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp/data/hbase/quota/.tabledesc/.tableinfo.0000000001 2023-07-24 20:11:06,011 INFO [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(7675): creating {ENCODED => 4fa2a21e5a49480a149a055b0fc39515, NAME => 'hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:quota', {NAME => 'q', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'u', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.tmp 2023-07-24 20:11:06,039 DEBUG [Listener at localhost/44145] zookeeper.ReadOnlyZKClient(139): Connect 0x65d0e58c to 127.0.0.1:62881 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-24 20:11:06,059 DEBUG [Listener at localhost/44145] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3763d38d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-24 20:11:06,061 DEBUG [hconnection-0x7d4b20e5-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-24 20:11:06,063 DEBUG [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(866): Instantiated hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:11:06,063 DEBUG [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(1604): Closing 4fa2a21e5a49480a149a055b0fc39515, disabling compactions & flushes 2023-07-24 20:11:06,063 INFO [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(1626): Closing region hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515. 2023-07-24 20:11:06,063 DEBUG [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515. 2023-07-24 20:11:06,063 DEBUG [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515. after waiting 0 ms 2023-07-24 20:11:06,064 DEBUG [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515. 2023-07-24 20:11:06,064 INFO [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(1838): Closed hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515. 2023-07-24 20:11:06,064 DEBUG [RegionOpenAndInit-hbase:quota-pool-0] regionserver.HRegion(1558): Region close journal for 4fa2a21e5a49480a149a055b0fc39515: 2023-07-24 20:11:06,064 INFO [RS-EventLoopGroup-10-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43030, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-24 20:11:06,067 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=119, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:quota execute state=CREATE_TABLE_ADD_TO_META 2023-07-24 20:11:06,068 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515.","families":{"info":[{"qualifier":"regioninfo","vlen":37,"tag":[],"timestamp":"1690229466068"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229466068"}]},"ts":"1690229466068"} 2023-07-24 20:11:06,070 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-24 20:11:06,070 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=119, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:quota execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-24 20:11:06,071 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:quota","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690229466070"}]},"ts":"1690229466070"} 2023-07-24 20:11:06,071 INFO [Listener at localhost/44145] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-07-24 20:11:06,071 DEBUG [Listener at localhost/44145] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x65d0e58c to 127.0.0.1:62881 2023-07-24 20:11:06,071 DEBUG [Listener at localhost/44145] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:11:06,073 INFO [Listener at localhost/44145] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase4.apache.org:33409 after: jenkins-hbase4.apache.org:33409 2023-07-24 20:11:06,073 DEBUG [Listener at localhost/44145] zookeeper.ReadOnlyZKClient(139): Connect 0x144d22d2 to 127.0.0.1:62881 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-24 20:11:06,073 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=hbase:quota, state=ENABLING in hbase:meta 2023-07-24 20:11:06,080 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-24 20:11:06,080 DEBUG [Listener at localhost/44145] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@302564b6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-24 20:11:06,080 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 20:11:06,081 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 20:11:06,081 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:11:06,081 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 20:11:06,081 DEBUG [PEWorker-2] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 20:11:06,083 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:quota, region=4fa2a21e5a49480a149a055b0fc39515, ASSIGN}] 2023-07-24 20:11:06,085 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:quota, region=4fa2a21e5a49480a149a055b0fc39515, ASSIGN 2023-07-24 20:11:06,085 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:quota, region=4fa2a21e5a49480a149a055b0fc39515, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,46065,1690229460063; forceNewPlan=false, retain=false 2023-07-24 20:11:06,236 INFO [jenkins-hbase4:33409] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-24 20:11:06,237 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=4fa2a21e5a49480a149a055b0fc39515, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46065,1690229460063 2023-07-24 20:11:06,237 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515.","families":{"info":[{"qualifier":"regioninfo","vlen":37,"tag":[],"timestamp":"1690229466237"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229466237"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229466237"}]},"ts":"1690229466237"} 2023-07-24 20:11:06,239 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=121, ppid=120, state=RUNNABLE; OpenRegionProcedure 4fa2a21e5a49480a149a055b0fc39515, server=jenkins-hbase4.apache.org,46065,1690229460063}] 2023-07-24 20:11:06,256 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-24 20:11:06,394 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515. 2023-07-24 20:11:06,394 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 4fa2a21e5a49480a149a055b0fc39515, NAME => 'hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515.', STARTKEY => '', ENDKEY => ''} 2023-07-24 20:11:06,395 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table quota 4fa2a21e5a49480a149a055b0fc39515 2023-07-24 20:11:06,395 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:11:06,395 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 4fa2a21e5a49480a149a055b0fc39515 2023-07-24 20:11:06,395 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 4fa2a21e5a49480a149a055b0fc39515 2023-07-24 20:11:06,396 INFO [StoreOpener-4fa2a21e5a49480a149a055b0fc39515-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family q of region 4fa2a21e5a49480a149a055b0fc39515 2023-07-24 20:11:06,398 DEBUG [StoreOpener-4fa2a21e5a49480a149a055b0fc39515-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/quota/4fa2a21e5a49480a149a055b0fc39515/q 2023-07-24 20:11:06,398 DEBUG [StoreOpener-4fa2a21e5a49480a149a055b0fc39515-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/quota/4fa2a21e5a49480a149a055b0fc39515/q 2023-07-24 20:11:06,398 INFO [StoreOpener-4fa2a21e5a49480a149a055b0fc39515-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4fa2a21e5a49480a149a055b0fc39515 columnFamilyName q 2023-07-24 20:11:06,399 INFO [StoreOpener-4fa2a21e5a49480a149a055b0fc39515-1] regionserver.HStore(310): Store=4fa2a21e5a49480a149a055b0fc39515/q, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:11:06,399 INFO [StoreOpener-4fa2a21e5a49480a149a055b0fc39515-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family u of region 4fa2a21e5a49480a149a055b0fc39515 2023-07-24 20:11:06,400 DEBUG [StoreOpener-4fa2a21e5a49480a149a055b0fc39515-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/quota/4fa2a21e5a49480a149a055b0fc39515/u 2023-07-24 20:11:06,400 DEBUG [StoreOpener-4fa2a21e5a49480a149a055b0fc39515-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/quota/4fa2a21e5a49480a149a055b0fc39515/u 2023-07-24 20:11:06,400 INFO [StoreOpener-4fa2a21e5a49480a149a055b0fc39515-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4fa2a21e5a49480a149a055b0fc39515 columnFamilyName u 2023-07-24 20:11:06,402 INFO [StoreOpener-4fa2a21e5a49480a149a055b0fc39515-1] regionserver.HStore(310): Store=4fa2a21e5a49480a149a055b0fc39515/u, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:11:06,403 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/quota/4fa2a21e5a49480a149a055b0fc39515 2023-07-24 20:11:06,404 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/quota/4fa2a21e5a49480a149a055b0fc39515 2023-07-24 20:11:06,405 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:quota descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 20:11:06,407 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 4fa2a21e5a49480a149a055b0fc39515 2023-07-24 20:11:06,412 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/quota/4fa2a21e5a49480a149a055b0fc39515/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 20:11:06,412 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 4fa2a21e5a49480a149a055b0fc39515; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=11912146560, jitterRate=0.10940510034561157}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 20:11:06,412 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 4fa2a21e5a49480a149a055b0fc39515: 2023-07-24 20:11:06,413 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515., pid=121, masterSystemTime=1690229466391 2023-07-24 20:11:06,415 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515. 2023-07-24 20:11:06,415 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515. 2023-07-24 20:11:06,415 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=4fa2a21e5a49480a149a055b0fc39515, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,46065,1690229460063 2023-07-24 20:11:06,415 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515.","families":{"info":[{"qualifier":"regioninfo","vlen":37,"tag":[],"timestamp":"1690229466415"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690229466415"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690229466415"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690229466415"}]},"ts":"1690229466415"} 2023-07-24 20:11:06,418 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=120 2023-07-24 20:11:06,419 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=120, state=SUCCESS; OpenRegionProcedure 4fa2a21e5a49480a149a055b0fc39515, server=jenkins-hbase4.apache.org,46065,1690229460063 in 178 msec 2023-07-24 20:11:06,420 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=119 2023-07-24 20:11:06,420 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=hbase:quota, region=4fa2a21e5a49480a149a055b0fc39515, ASSIGN in 338 msec 2023-07-24 20:11:06,421 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=119, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:quota execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-24 20:11:06,421 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:quota","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690229466421"}]},"ts":"1690229466421"} 2023-07-24 20:11:06,422 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:quota, state=ENABLED in hbase:meta 2023-07-24 20:11:06,425 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=119, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:quota execute state=CREATE_TABLE_POST_OPERATION 2023-07-24 20:11:06,426 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; CreateTableProcedure table=hbase:quota in 459 msec 2023-07-24 20:11:06,489 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-24 20:11:06,490 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:rsgroup' 2023-07-24 20:11:06,490 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-24 20:11:06,491 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:quota' 2023-07-24 20:11:07,892 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: Master,sub=Coprocessor.Master.CP_org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase$CPMasterObserver 2023-07-24 20:11:07,892 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering Master,sub=Coprocessor.Master.CP_org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase$CPMasterObserver Metrics about HBase MasterObservers 2023-07-24 20:11:07,892 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: Master,sub=Coprocessor.Master.CP_org.apache.hadoop.hbase.quotas.MasterQuotasObserver 2023-07-24 20:11:07,892 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering Master,sub=Coprocessor.Master.CP_org.apache.hadoop.hbase.quotas.MasterQuotasObserver Metrics about HBase MasterObservers 2023-07-24 20:11:07,892 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-24 20:11:07,892 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-07-24 20:11:07,892 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: Master,sub=Coprocessor.Master.CP_org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint 2023-07-24 20:11:07,892 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering Master,sub=Coprocessor.Master.CP_org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint Metrics about HBase MasterObservers 2023-07-24 20:11:09,181 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33409,1690229459778] rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker(840): RSGroup table=hbase:rsgroup is online, refreshing cached information 2023-07-24 20:11:09,181 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33409,1690229459778] rsgroup.RSGroupInfoManagerImpl(534): Refreshing in Online mode. 2023-07-24 20:11:09,187 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33409,1690229459778] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:11:09,188 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33409,1690229459778] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:11:09,188 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33409,1690229459778] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-24 20:11:09,189 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rsgroup 2023-07-24 20:11:09,189 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,33409,1690229459778] rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker(823): GroupBasedLoadBalancer is now online 2023-07-24 20:11:09,286 DEBUG [Listener at localhost/44145] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-07-24 20:11:09,287 INFO [RS-EventLoopGroup-9-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60324, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-07-24 20:11:09,289 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-24 20:11:09,289 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=33409] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-07-24 20:11:09,291 DEBUG [Listener at localhost/44145] zookeeper.ReadOnlyZKClient(139): Connect 0x4700ebae to 127.0.0.1:62881 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-24 20:11:09,297 DEBUG [Listener at localhost/44145] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@549213d0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-24 20:11:09,297 INFO [Listener at localhost/44145] zookeeper.RecoverableZooKeeper(93): Process identifier=VerifyingRSGroupAdminClient connecting to ZooKeeper ensemble=127.0.0.1:62881 2023-07-24 20:11:09,298 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [90,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:11:09,299 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): VerifyingRSGroupAdminClient0x0, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-24 20:11:09,300 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): VerifyingRSGroupAdminClient-0x10198f4f3ce001b connected 2023-07-24 20:11:09,301 DEBUG [Listener at localhost/44145] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-24 20:11:09,302 INFO [RS-EventLoopGroup-10-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43042, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-24 20:11:09,309 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBasics(309): Shutting down cluster 2023-07-24 20:11:09,309 INFO [Listener at localhost/44145] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-07-24 20:11:09,309 DEBUG [Listener at localhost/44145] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x144d22d2 to 127.0.0.1:62881 2023-07-24 20:11:09,309 DEBUG [Listener at localhost/44145] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:11:09,310 DEBUG [Listener at localhost/44145] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-24 20:11:09,310 DEBUG [Listener at localhost/44145] util.JVMClusterUtil(257): Found active master hash=1773006880, stopped=false 2023-07-24 20:11:09,310 DEBUG [Listener at localhost/44145] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint 2023-07-24 20:11:09,310 DEBUG [Listener at localhost/44145] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase$CPMasterObserver 2023-07-24 20:11:09,310 DEBUG [Listener at localhost/44145] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.quotas.MasterQuotasObserver 2023-07-24 20:11:09,310 INFO [Listener at localhost/44145] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,33409,1690229459778 2023-07-24 20:11:09,311 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-24 20:11:09,311 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:46705-0x10198f4f3ce0011, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-24 20:11:09,311 INFO [Listener at localhost/44145] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 20:11:09,311 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 20:11:09,311 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:45019-0x10198f4f3ce0012, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-24 20:11:09,311 DEBUG [Listener at localhost/44145] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4d670eec to 127.0.0.1:62881 2023-07-24 20:11:09,311 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:46065-0x10198f4f3ce0013, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-24 20:11:09,312 DEBUG [Listener at localhost/44145] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:11:09,311 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 20:11:09,312 INFO [Listener at localhost/44145] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,46705,1690229459880' ***** 2023-07-24 20:11:09,312 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46705-0x10198f4f3ce0011, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 20:11:09,312 INFO [Listener at localhost/44145] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-24 20:11:09,312 INFO [Listener at localhost/44145] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,45019,1690229459931' ***** 2023-07-24 20:11:09,312 INFO [Listener at localhost/44145] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-24 20:11:09,312 INFO [Listener at localhost/44145] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,46065,1690229460063' ***** 2023-07-24 20:11:09,312 INFO [Listener at localhost/44145] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-24 20:11:09,312 INFO [RS:1;jenkins-hbase4:45019] regionserver.HRegionServer(1109): Stopping infoServer 2023-07-24 20:11:09,312 INFO [RS:0;jenkins-hbase4:46705] regionserver.HRegionServer(1109): Stopping infoServer 2023-07-24 20:11:09,312 INFO [RS:2;jenkins-hbase4:46065] regionserver.HRegionServer(1109): Stopping infoServer 2023-07-24 20:11:09,317 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45019-0x10198f4f3ce0012, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 20:11:09,319 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-24 20:11:09,319 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-24 20:11:09,319 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-24 20:11:09,321 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-24 20:11:09,319 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-24 20:11:09,319 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-24 20:11:09,323 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46065-0x10198f4f3ce0013, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 20:11:09,324 INFO [RS:1;jenkins-hbase4:45019] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.w.WebAppContext@55838389{regionserver,/,null,STOPPED}{file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver} 2023-07-24 20:11:09,324 INFO [RS:2;jenkins-hbase4:46065] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.w.WebAppContext@68e950a2{regionserver,/,null,STOPPED}{file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver} 2023-07-24 20:11:09,324 INFO [RS:0;jenkins-hbase4:46705] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.w.WebAppContext@722a21a7{regionserver,/,null,STOPPED}{file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver} 2023-07-24 20:11:09,325 INFO [RS:1;jenkins-hbase4:45019] server.AbstractConnector(383): Stopped ServerConnector@64a829ad{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-24 20:11:09,325 INFO [RS:0;jenkins-hbase4:46705] server.AbstractConnector(383): Stopped ServerConnector@f3c8deb{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-24 20:11:09,325 INFO [RS:2;jenkins-hbase4:46065] server.AbstractConnector(383): Stopped ServerConnector@63c9e904{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-24 20:11:09,325 INFO [RS:0;jenkins-hbase4:46705] session.HouseKeeper(149): node0 Stopped scavenging 2023-07-24 20:11:09,325 INFO [RS:1;jenkins-hbase4:45019] session.HouseKeeper(149): node0 Stopped scavenging 2023-07-24 20:11:09,325 INFO [RS:2;jenkins-hbase4:46065] session.HouseKeeper(149): node0 Stopped scavenging 2023-07-24 20:11:09,325 INFO [RS:0;jenkins-hbase4:46705] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@1a6b043d{static,/static,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/static/,STOPPED} 2023-07-24 20:11:09,325 INFO [RS:1;jenkins-hbase4:45019] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@53cc2d4e{static,/static,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/static/,STOPPED} 2023-07-24 20:11:09,325 INFO [RS:0;jenkins-hbase4:46705] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@14daa73{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/hadoop.log.dir/,STOPPED} 2023-07-24 20:11:09,325 INFO [RS:2;jenkins-hbase4:46065] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@7f5a1268{static,/static,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/static/,STOPPED} 2023-07-24 20:11:09,325 INFO [RS:1;jenkins-hbase4:45019] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@38e22bc7{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/hadoop.log.dir/,STOPPED} 2023-07-24 20:11:09,326 INFO [RS:2;jenkins-hbase4:46065] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@606f4bdb{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/hadoop.log.dir/,STOPPED} 2023-07-24 20:11:09,326 INFO [RS:0;jenkins-hbase4:46705] regionserver.HeapMemoryManager(220): Stopping 2023-07-24 20:11:09,326 INFO [RS:0;jenkins-hbase4:46705] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-24 20:11:09,327 INFO [RS:2;jenkins-hbase4:46065] regionserver.HeapMemoryManager(220): Stopping 2023-07-24 20:11:09,327 INFO [RS:1;jenkins-hbase4:45019] regionserver.HeapMemoryManager(220): Stopping 2023-07-24 20:11:09,327 INFO [RS:0;jenkins-hbase4:46705] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-24 20:11:09,327 INFO [RS:1;jenkins-hbase4:45019] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-24 20:11:09,327 INFO [RS:0;jenkins-hbase4:46705] regionserver.HRegionServer(3305): Received CLOSE for ec4f892f95beff2c1aa7863da43a18fa 2023-07-24 20:11:09,327 INFO [RS:2;jenkins-hbase4:46065] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-24 20:11:09,327 INFO [RS:1;jenkins-hbase4:45019] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-24 20:11:09,327 INFO [RS:2;jenkins-hbase4:46065] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-24 20:11:09,327 INFO [RS:2;jenkins-hbase4:46065] regionserver.HRegionServer(3305): Received CLOSE for 4fa2a21e5a49480a149a055b0fc39515 2023-07-24 20:11:09,327 INFO [RS:1;jenkins-hbase4:45019] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,45019,1690229459931 2023-07-24 20:11:09,327 DEBUG [RS:1;jenkins-hbase4:45019] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4b85d299 to 127.0.0.1:62881 2023-07-24 20:11:09,327 DEBUG [RS:1;jenkins-hbase4:45019] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:11:09,327 INFO [RS:1;jenkins-hbase4:45019] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,45019,1690229459931; all regions closed. 2023-07-24 20:11:09,328 DEBUG [RS:1;jenkins-hbase4:45019] quotas.QuotaCache(100): Stopping QuotaRefresherChore chore. 2023-07-24 20:11:09,328 INFO [RS:0;jenkins-hbase4:46705] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,46705,1690229459880 2023-07-24 20:11:09,328 DEBUG [RS:0;jenkins-hbase4:46705] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x104fb07d to 127.0.0.1:62881 2023-07-24 20:11:09,328 DEBUG [RS:0;jenkins-hbase4:46705] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:11:09,328 INFO [RS:0;jenkins-hbase4:46705] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-24 20:11:09,328 INFO [RS:0;jenkins-hbase4:46705] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-24 20:11:09,328 INFO [RS:0;jenkins-hbase4:46705] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-24 20:11:09,328 INFO [RS:0;jenkins-hbase4:46705] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-24 20:11:09,328 INFO [RS:2;jenkins-hbase4:46065] regionserver.HRegionServer(3305): Received CLOSE for 324410cf2a29c2dca1a19e8570799d47 2023-07-24 20:11:09,328 INFO [RS:2;jenkins-hbase4:46065] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,46065,1690229460063 2023-07-24 20:11:09,328 DEBUG [RS:2;jenkins-hbase4:46065] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x482a57c8 to 127.0.0.1:62881 2023-07-24 20:11:09,328 DEBUG [RS:2;jenkins-hbase4:46065] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:11:09,328 INFO [RS:2;jenkins-hbase4:46065] regionserver.HRegionServer(1474): Waiting on 2 regions to close 2023-07-24 20:11:09,328 DEBUG [RS:2;jenkins-hbase4:46065] regionserver.HRegionServer(1478): Online Regions={4fa2a21e5a49480a149a055b0fc39515=hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515., 324410cf2a29c2dca1a19e8570799d47=hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47.} 2023-07-24 20:11:09,329 DEBUG [RS:2;jenkins-hbase4:46065] regionserver.HRegionServer(1504): Waiting on 324410cf2a29c2dca1a19e8570799d47, 4fa2a21e5a49480a149a055b0fc39515 2023-07-24 20:11:09,329 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 4fa2a21e5a49480a149a055b0fc39515, disabling compactions & flushes 2023-07-24 20:11:09,330 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-24 20:11:09,330 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515. 2023-07-24 20:11:09,330 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-24 20:11:09,329 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ec4f892f95beff2c1aa7863da43a18fa, disabling compactions & flushes 2023-07-24 20:11:09,329 INFO [RS:0;jenkins-hbase4:46705] regionserver.HRegionServer(1474): Waiting on 2 regions to close 2023-07-24 20:11:09,330 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. 2023-07-24 20:11:09,330 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-24 20:11:09,330 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515. 2023-07-24 20:11:09,330 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-24 20:11:09,330 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-24 20:11:09,330 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. 2023-07-24 20:11:09,331 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=3.05 KB heapSize=5.87 KB 2023-07-24 20:11:09,331 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. after waiting 0 ms 2023-07-24 20:11:09,330 DEBUG [RS:0;jenkins-hbase4:46705] regionserver.HRegionServer(1478): Online Regions={1588230740=hbase:meta,,1.1588230740, ec4f892f95beff2c1aa7863da43a18fa=hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa.} 2023-07-24 20:11:09,331 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. 2023-07-24 20:11:09,331 DEBUG [RS:0;jenkins-hbase4:46705] regionserver.HRegionServer(1504): Waiting on 1588230740, ec4f892f95beff2c1aa7863da43a18fa 2023-07-24 20:11:09,330 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515. after waiting 0 ms 2023-07-24 20:11:09,331 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing ec4f892f95beff2c1aa7863da43a18fa 1/1 column families, dataSize=242 B heapSize=648 B 2023-07-24 20:11:09,331 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515. 2023-07-24 20:11:09,342 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/quota/4fa2a21e5a49480a149a055b0fc39515/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-24 20:11:09,342 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515. 2023-07-24 20:11:09,343 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 4fa2a21e5a49480a149a055b0fc39515: 2023-07-24 20:11:09,343 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515. 2023-07-24 20:11:09,347 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 324410cf2a29c2dca1a19e8570799d47, disabling compactions & flushes 2023-07-24 20:11:09,347 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. 2023-07-24 20:11:09,347 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. 2023-07-24 20:11:09,347 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. after waiting 0 ms 2023-07-24 20:11:09,347 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. 2023-07-24 20:11:09,349 DEBUG [RS:1;jenkins-hbase4:45019] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/oldWALs 2023-07-24 20:11:09,349 INFO [RS:1;jenkins-hbase4:45019] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C45019%2C1690229459931:(num 1690229460645) 2023-07-24 20:11:09,349 DEBUG [RS:1;jenkins-hbase4:45019] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:11:09,349 INFO [RS:1;jenkins-hbase4:45019] regionserver.LeaseManager(133): Closed leases 2023-07-24 20:11:09,350 INFO [RS:1;jenkins-hbase4:45019] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-24 20:11:09,350 INFO [RS:1;jenkins-hbase4:45019] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-24 20:11:09,350 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-24 20:11:09,350 INFO [RS:1;jenkins-hbase4:45019] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-24 20:11:09,350 INFO [RS:1;jenkins-hbase4:45019] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-24 20:11:09,351 INFO [RS:1;jenkins-hbase4:45019] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:45019 2023-07-24 20:11:09,355 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/namespace/324410cf2a29c2dca1a19e8570799d47/recovered.edits/17.seqid, newMaxSeqId=17, maxSeqId=14 2023-07-24 20:11:09,358 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. 2023-07-24 20:11:09,358 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 324410cf2a29c2dca1a19e8570799d47: 2023-07-24 20:11:09,358 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. 2023-07-24 20:11:09,360 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=242 B at sequenceid=74 (bloomFilter=true), to=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/rsgroup/ec4f892f95beff2c1aa7863da43a18fa/.tmp/m/cfb13056081245e68f9511bc8b4d0363 2023-07-24 20:11:09,361 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.97 KB at sequenceid=155 (bloomFilter=false), to=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/.tmp/info/99bce8ea8bc2478680bf62bc041d6208 2023-07-24 20:11:09,361 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:46065-0x10198f4f3ce0013, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,45019,1690229459931 2023-07-24 20:11:09,361 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:11:09,361 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:46065-0x10198f4f3ce0013, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:11:09,362 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:45019-0x10198f4f3ce0012, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,45019,1690229459931 2023-07-24 20:11:09,361 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:46705-0x10198f4f3ce0011, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,45019,1690229459931 2023-07-24 20:11:09,362 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:45019-0x10198f4f3ce0012, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:11:09,362 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:46705-0x10198f4f3ce0011, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:11:09,362 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,45019,1690229459931] 2023-07-24 20:11:09,362 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,45019,1690229459931; numProcessing=1 2023-07-24 20:11:09,363 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,45019,1690229459931 already deleted, retry=false 2023-07-24 20:11:09,364 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,45019,1690229459931 expired; onlineServers=2 2023-07-24 20:11:09,367 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/rsgroup/ec4f892f95beff2c1aa7863da43a18fa/.tmp/m/cfb13056081245e68f9511bc8b4d0363 as hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/rsgroup/ec4f892f95beff2c1aa7863da43a18fa/m/cfb13056081245e68f9511bc8b4d0363 2023-07-24 20:11:09,374 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/rsgroup/ec4f892f95beff2c1aa7863da43a18fa/m/cfb13056081245e68f9511bc8b4d0363, entries=2, sequenceid=74, filesize=5.0 K 2023-07-24 20:11:09,376 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~242 B/242, heapSize ~632 B/632, currentSize=0 B/0 for ec4f892f95beff2c1aa7863da43a18fa in 45ms, sequenceid=74, compaction requested=false 2023-07-24 20:11:09,386 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=86 B at sequenceid=155 (bloomFilter=false), to=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/.tmp/table/228b116fcd95447abde6f2baeea5c266 2023-07-24 20:11:09,387 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/rsgroup/ec4f892f95beff2c1aa7863da43a18fa/recovered.edits/77.seqid, newMaxSeqId=77, maxSeqId=70 2023-07-24 20:11:09,387 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-24 20:11:09,388 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. 2023-07-24 20:11:09,388 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ec4f892f95beff2c1aa7863da43a18fa: 2023-07-24 20:11:09,388 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. 2023-07-24 20:11:09,393 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/.tmp/info/99bce8ea8bc2478680bf62bc041d6208 as hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/info/99bce8ea8bc2478680bf62bc041d6208 2023-07-24 20:11:09,399 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/info/99bce8ea8bc2478680bf62bc041d6208, entries=26, sequenceid=155, filesize=7.7 K 2023-07-24 20:11:09,399 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/.tmp/table/228b116fcd95447abde6f2baeea5c266 as hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/table/228b116fcd95447abde6f2baeea5c266 2023-07-24 20:11:09,405 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/table/228b116fcd95447abde6f2baeea5c266, entries=2, sequenceid=155, filesize=4.7 K 2023-07-24 20:11:09,406 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~3.05 KB/3126, heapSize ~5.59 KB/5720, currentSize=0 B/0 for 1588230740 in 76ms, sequenceid=155, compaction requested=true 2023-07-24 20:11:09,419 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/recovered.edits/158.seqid, newMaxSeqId=158, maxSeqId=143 2023-07-24 20:11:09,419 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-24 20:11:09,420 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-24 20:11:09,420 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-24 20:11:09,420 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-24 20:11:09,512 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:45019-0x10198f4f3ce0012, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 20:11:09,512 INFO [RS:1;jenkins-hbase4:45019] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,45019,1690229459931; zookeeper connection closed. 2023-07-24 20:11:09,512 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:45019-0x10198f4f3ce0012, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 20:11:09,513 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-07-24 20:11:09,513 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@408531fb] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@408531fb 2023-07-24 20:11:09,513 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-07-24 20:11:09,513 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-07-24 20:11:09,514 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-07-24 20:11:09,529 INFO [RS:2;jenkins-hbase4:46065] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,46065,1690229460063; all regions closed. 2023-07-24 20:11:09,529 DEBUG [RS:2;jenkins-hbase4:46065] quotas.QuotaCache(100): Stopping QuotaRefresherChore chore. 2023-07-24 20:11:09,531 INFO [RS:0;jenkins-hbase4:46705] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,46705,1690229459880; all regions closed. 2023-07-24 20:11:09,531 DEBUG [RS:0;jenkins-hbase4:46705] quotas.QuotaCache(100): Stopping QuotaRefresherChore chore. 2023-07-24 20:11:09,536 DEBUG [RS:2;jenkins-hbase4:46065] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/oldWALs 2023-07-24 20:11:09,536 INFO [RS:2;jenkins-hbase4:46065] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C46065%2C1690229460063:(num 1690229460652) 2023-07-24 20:11:09,536 DEBUG [RS:2;jenkins-hbase4:46065] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:11:09,536 INFO [RS:2;jenkins-hbase4:46065] regionserver.LeaseManager(133): Closed leases 2023-07-24 20:11:09,537 INFO [RS:2;jenkins-hbase4:46065] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-24 20:11:09,537 INFO [RS:2;jenkins-hbase4:46065] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-24 20:11:09,537 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-24 20:11:09,537 INFO [RS:2;jenkins-hbase4:46065] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-24 20:11:09,537 INFO [RS:2;jenkins-hbase4:46065] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-24 20:11:09,538 INFO [RS:2;jenkins-hbase4:46065] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:46065 2023-07-24 20:11:09,539 DEBUG [RS:0;jenkins-hbase4:46705] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/oldWALs 2023-07-24 20:11:09,539 INFO [RS:0;jenkins-hbase4:46705] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C46705%2C1690229459880.meta:.meta(num 1690229460732) 2023-07-24 20:11:09,540 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:46705-0x10198f4f3ce0011, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,46065,1690229460063 2023-07-24 20:11:09,540 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:46065-0x10198f4f3ce0013, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,46065,1690229460063 2023-07-24 20:11:09,540 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:11:09,549 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,46065,1690229460063] 2023-07-24 20:11:09,549 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,46065,1690229460063; numProcessing=2 2023-07-24 20:11:09,549 DEBUG [RS:0;jenkins-hbase4:46705] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/oldWALs 2023-07-24 20:11:09,549 INFO [RS:0;jenkins-hbase4:46705] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C46705%2C1690229459880:(num 1690229460652) 2023-07-24 20:11:09,549 DEBUG [RS:0;jenkins-hbase4:46705] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:11:09,549 INFO [RS:0;jenkins-hbase4:46705] regionserver.LeaseManager(133): Closed leases 2023-07-24 20:11:09,550 INFO [RS:0;jenkins-hbase4:46705] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-24 20:11:09,550 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-24 20:11:09,550 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,46065,1690229460063 already deleted, retry=false 2023-07-24 20:11:09,550 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,46065,1690229460063 expired; onlineServers=1 2023-07-24 20:11:09,550 INFO [RS:0;jenkins-hbase4:46705] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:46705 2023-07-24 20:11:09,554 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:46705-0x10198f4f3ce0011, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,46705,1690229459880 2023-07-24 20:11:09,554 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:11:09,556 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,46705,1690229459880] 2023-07-24 20:11:09,556 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,46705,1690229459880; numProcessing=3 2023-07-24 20:11:09,557 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,46705,1690229459880 already deleted, retry=false 2023-07-24 20:11:09,557 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,46705,1690229459880 expired; onlineServers=0 2023-07-24 20:11:09,557 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,33409,1690229459778' ***** 2023-07-24 20:11:09,557 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-24 20:11:09,558 DEBUG [M:0;jenkins-hbase4:33409] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6d9269cd, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-24 20:11:09,558 INFO [M:0;jenkins-hbase4:33409] regionserver.HRegionServer(1109): Stopping infoServer 2023-07-24 20:11:09,560 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-07-24 20:11:09,560 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 20:11:09,560 INFO [M:0;jenkins-hbase4:33409] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.w.WebAppContext@1f8ba06{master,/,null,STOPPED}{file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/master} 2023-07-24 20:11:09,560 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-24 20:11:09,560 INFO [M:0;jenkins-hbase4:33409] server.AbstractConnector(383): Stopped ServerConnector@58b84ff5{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-24 20:11:09,560 INFO [M:0;jenkins-hbase4:33409] session.HouseKeeper(149): node0 Stopped scavenging 2023-07-24 20:11:09,560 INFO [M:0;jenkins-hbase4:33409] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@726dac4c{static,/static,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/static/,STOPPED} 2023-07-24 20:11:09,560 INFO [M:0;jenkins-hbase4:33409] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@fa245c3{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/hadoop.log.dir/,STOPPED} 2023-07-24 20:11:09,561 INFO [M:0;jenkins-hbase4:33409] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,33409,1690229459778 2023-07-24 20:11:09,561 INFO [M:0;jenkins-hbase4:33409] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,33409,1690229459778; all regions closed. 2023-07-24 20:11:09,561 DEBUG [M:0;jenkins-hbase4:33409] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:11:09,561 INFO [M:0;jenkins-hbase4:33409] master.HMaster(1491): Stopping master jetty server 2023-07-24 20:11:09,562 INFO [M:0;jenkins-hbase4:33409] server.AbstractConnector(383): Stopped ServerConnector@194f0464{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-24 20:11:09,562 DEBUG [M:0;jenkins-hbase4:33409] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-24 20:11:09,563 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-24 20:11:09,563 DEBUG [M:0;jenkins-hbase4:33409] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-24 20:11:09,563 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1690229460400] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1690229460400,5,FailOnTimeoutGroup] 2023-07-24 20:11:09,563 INFO [M:0;jenkins-hbase4:33409] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-24 20:11:09,564 INFO [M:0;jenkins-hbase4:33409] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-24 20:11:09,563 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1690229460400] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1690229460400,5,FailOnTimeoutGroup] 2023-07-24 20:11:09,564 INFO [M:0;jenkins-hbase4:33409] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [ScheduledChore name=QuotaObserverChore, period=60000, unit=MILLISECONDS] on shutdown 2023-07-24 20:11:09,564 DEBUG [M:0;jenkins-hbase4:33409] master.HMaster(1512): Stopping service threads 2023-07-24 20:11:09,564 INFO [M:0;jenkins-hbase4:33409] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-24 20:11:09,565 ERROR [M:0;jenkins-hbase4:33409] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 20:11:09,565 INFO [M:0;jenkins-hbase4:33409] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 20:11:09,565 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-24 20:11:09,565 DEBUG [M:0;jenkins-hbase4:33409] zookeeper.ZKUtil(398): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-07-24 20:11:09,565 WARN [M:0;jenkins-hbase4:33409] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-24 20:11:09,565 INFO [M:0;jenkins-hbase4:33409] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 20:11:09,566 INFO [M:0;jenkins-hbase4:33409] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-24 20:11:09,566 DEBUG [M:0;jenkins-hbase4:33409] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-24 20:11:09,567 INFO [M:0;jenkins-hbase4:33409] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-24 20:11:09,567 DEBUG [M:0;jenkins-hbase4:33409] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-24 20:11:09,567 DEBUG [M:0;jenkins-hbase4:33409] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-24 20:11:09,567 DEBUG [M:0;jenkins-hbase4:33409] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-24 20:11:09,567 INFO [M:0;jenkins-hbase4:33409] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=44.95 KB heapSize=54.44 KB 2023-07-24 20:11:09,580 INFO [M:0;jenkins-hbase4:33409] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=44.95 KB at sequenceid=909 (bloomFilter=true), to=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/273705b3a8934e9f89526eacd88c3946 2023-07-24 20:11:09,585 DEBUG [M:0;jenkins-hbase4:33409] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/273705b3a8934e9f89526eacd88c3946 as hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/273705b3a8934e9f89526eacd88c3946 2023-07-24 20:11:09,591 INFO [M:0;jenkins-hbase4:33409] regionserver.HStore(1080): Added hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/273705b3a8934e9f89526eacd88c3946, entries=13, sequenceid=909, filesize=7.2 K 2023-07-24 20:11:09,592 INFO [M:0;jenkins-hbase4:33409] regionserver.HRegion(2948): Finished flush of dataSize ~44.95 KB/46033, heapSize ~54.42 KB/55728, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 24ms, sequenceid=909, compaction requested=false 2023-07-24 20:11:09,595 INFO [M:0;jenkins-hbase4:33409] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-24 20:11:09,595 DEBUG [M:0;jenkins-hbase4:33409] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-24 20:11:09,598 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/WALs/jenkins-hbase4.apache.org,33409,1690229459778/jenkins-hbase4.apache.org%2C33409%2C1690229459778.1690229460221 not finished, retry = 0 2023-07-24 20:11:09,699 INFO [M:0;jenkins-hbase4:33409] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-24 20:11:09,699 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-24 20:11:09,700 INFO [M:0;jenkins-hbase4:33409] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33409 2023-07-24 20:11:09,702 DEBUG [M:0;jenkins-hbase4:33409] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,33409,1690229459778 already deleted, retry=false 2023-07-24 20:11:10,013 INFO [M:0;jenkins-hbase4:33409] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,33409,1690229459778; zookeeper connection closed. 2023-07-24 20:11:10,013 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 20:11:10,014 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:33409-0x10198f4f3ce0010, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 20:11:10,113 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:46705-0x10198f4f3ce0011, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 20:11:10,113 INFO [RS:0;jenkins-hbase4:46705] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,46705,1690229459880; zookeeper connection closed. 2023-07-24 20:11:10,114 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:46705-0x10198f4f3ce0011, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 20:11:10,114 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@21ee9700] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@21ee9700 2023-07-24 20:11:10,214 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:46065-0x10198f4f3ce0013, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 20:11:10,214 INFO [RS:2;jenkins-hbase4:46065] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,46065,1690229460063; zookeeper connection closed. 2023-07-24 20:11:10,214 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:46065-0x10198f4f3ce0013, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 20:11:10,214 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@3da486c2] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@3da486c2 2023-07-24 20:11:10,214 INFO [Listener at localhost/44145] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-07-24 20:11:10,214 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBasics(311): Sleeping a bit 2023-07-24 20:11:11,899 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-24 20:11:12,216 INFO [Listener at localhost/44145] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-07-24 20:11:12,216 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 20:11:12,217 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-24 20:11:12,217 INFO [Listener at localhost/44145] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-24 20:11:12,217 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 20:11:12,217 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-24 20:11:12,217 INFO [Listener at localhost/44145] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-24 20:11:12,218 INFO [Listener at localhost/44145] ipc.NettyRpcServer(120): Bind to /172.31.14.131:37489 2023-07-24 20:11:12,218 INFO [Listener at localhost/44145] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:11:12,219 INFO [Listener at localhost/44145] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:11:12,220 INFO [Listener at localhost/44145] zookeeper.RecoverableZooKeeper(93): Process identifier=master:37489 connecting to ZooKeeper ensemble=127.0.0.1:62881 2023-07-24 20:11:12,223 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:374890x0, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-24 20:11:12,224 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:37489-0x10198f4f3ce001c connected 2023-07-24 20:11:12,227 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-24 20:11:12,227 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 20:11:12,228 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-24 20:11:12,229 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37489 2023-07-24 20:11:12,229 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37489 2023-07-24 20:11:12,232 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37489 2023-07-24 20:11:12,233 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37489 2023-07-24 20:11:12,233 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37489 2023-07-24 20:11:12,236 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2023-07-24 20:11:12,236 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter) 2023-07-24 20:11:12,236 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'securityheaders' (class=org.apache.hadoop.hbase.http.SecurityHeadersFilter) 2023-07-24 20:11:12,236 INFO [Listener at localhost/44145] http.HttpServer(879): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context master 2023-07-24 20:11:12,237 INFO [Listener at localhost/44145] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2023-07-24 20:11:12,237 INFO [Listener at localhost/44145] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2023-07-24 20:11:12,237 INFO [Listener at localhost/44145] http.HttpServer(783): ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint. 2023-07-24 20:11:12,238 INFO [Listener at localhost/44145] http.HttpServer(1146): Jetty bound to port 43499 2023-07-24 20:11:12,238 INFO [Listener at localhost/44145] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-24 20:11:12,242 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:11:12,243 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@1e7ef621{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/hadoop.log.dir/,AVAILABLE} 2023-07-24 20:11:12,243 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:11:12,243 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@57548a1b{static,/static,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/static/,AVAILABLE} 2023-07-24 20:11:12,249 INFO [Listener at localhost/44145] webapp.StandardDescriptorProcessor(277): NO JSP Support for /, did not find org.apache.hbase.thirdparty.org.eclipse.jetty.jsp.JettyJspServlet 2023-07-24 20:11:12,250 INFO [Listener at localhost/44145] session.DefaultSessionIdManager(334): DefaultSessionIdManager workerName=node0 2023-07-24 20:11:12,250 INFO [Listener at localhost/44145] session.DefaultSessionIdManager(339): No SessionScavenger set, using defaults 2023-07-24 20:11:12,250 INFO [Listener at localhost/44145] session.HouseKeeper(132): node0 Scavenging every 660000ms 2023-07-24 20:11:12,252 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:11:12,253 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.w.WebAppContext@71da3018{master,/,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/master/,AVAILABLE}{file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/master} 2023-07-24 20:11:12,254 INFO [Listener at localhost/44145] server.AbstractConnector(333): Started ServerConnector@49f3f2e8{HTTP/1.1, (http/1.1)}{0.0.0.0:43499} 2023-07-24 20:11:12,254 INFO [Listener at localhost/44145] server.Server(415): Started @40821ms 2023-07-24 20:11:12,254 INFO [Listener at localhost/44145] master.HMaster(444): hbase.rootdir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4, hbase.cluster.distributed=false 2023-07-24 20:11:12,256 DEBUG [pool-521-thread-1] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: INIT 2023-07-24 20:11:12,273 INFO [Listener at localhost/44145] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-24 20:11:12,274 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 20:11:12,274 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-24 20:11:12,274 INFO [Listener at localhost/44145] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-24 20:11:12,274 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 20:11:12,274 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-24 20:11:12,274 INFO [Listener at localhost/44145] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-24 20:11:12,275 INFO [Listener at localhost/44145] ipc.NettyRpcServer(120): Bind to /172.31.14.131:46693 2023-07-24 20:11:12,275 INFO [Listener at localhost/44145] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-24 20:11:12,278 DEBUG [Listener at localhost/44145] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-24 20:11:12,279 INFO [Listener at localhost/44145] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:11:12,281 INFO [Listener at localhost/44145] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:11:12,282 INFO [Listener at localhost/44145] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:46693 connecting to ZooKeeper ensemble=127.0.0.1:62881 2023-07-24 20:11:12,286 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:466930x0, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-24 20:11:12,287 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): regionserver:466930x0, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-24 20:11:12,288 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): regionserver:466930x0, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 20:11:12,288 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:46693-0x10198f4f3ce001d connected 2023-07-24 20:11:12,289 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): regionserver:46693-0x10198f4f3ce001d, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-24 20:11:12,295 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=46693 2023-07-24 20:11:12,296 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=46693 2023-07-24 20:11:12,296 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=46693 2023-07-24 20:11:12,297 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=46693 2023-07-24 20:11:12,299 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=46693 2023-07-24 20:11:12,301 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2023-07-24 20:11:12,301 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter) 2023-07-24 20:11:12,301 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'securityheaders' (class=org.apache.hadoop.hbase.http.SecurityHeadersFilter) 2023-07-24 20:11:12,302 INFO [Listener at localhost/44145] http.HttpServer(879): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context regionserver 2023-07-24 20:11:12,302 INFO [Listener at localhost/44145] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2023-07-24 20:11:12,302 INFO [Listener at localhost/44145] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2023-07-24 20:11:12,302 INFO [Listener at localhost/44145] http.HttpServer(783): ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint. 2023-07-24 20:11:12,303 INFO [Listener at localhost/44145] http.HttpServer(1146): Jetty bound to port 43467 2023-07-24 20:11:12,303 INFO [Listener at localhost/44145] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-24 20:11:12,311 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:11:12,311 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@750dea8b{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/hadoop.log.dir/,AVAILABLE} 2023-07-24 20:11:12,311 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:11:12,312 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@dfee794{static,/static,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/static/,AVAILABLE} 2023-07-24 20:11:12,319 INFO [Listener at localhost/44145] webapp.StandardDescriptorProcessor(277): NO JSP Support for /, did not find org.apache.hbase.thirdparty.org.eclipse.jetty.jsp.JettyJspServlet 2023-07-24 20:11:12,319 INFO [Listener at localhost/44145] session.DefaultSessionIdManager(334): DefaultSessionIdManager workerName=node0 2023-07-24 20:11:12,320 INFO [Listener at localhost/44145] session.DefaultSessionIdManager(339): No SessionScavenger set, using defaults 2023-07-24 20:11:12,320 INFO [Listener at localhost/44145] session.HouseKeeper(132): node0 Scavenging every 660000ms 2023-07-24 20:11:12,321 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:11:12,322 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.w.WebAppContext@5c3d90e7{regionserver,/,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver/,AVAILABLE}{file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver} 2023-07-24 20:11:12,324 INFO [Listener at localhost/44145] server.AbstractConnector(333): Started ServerConnector@237e24d4{HTTP/1.1, (http/1.1)}{0.0.0.0:43467} 2023-07-24 20:11:12,324 INFO [Listener at localhost/44145] server.Server(415): Started @40891ms 2023-07-24 20:11:12,336 INFO [Listener at localhost/44145] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-24 20:11:12,336 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 20:11:12,336 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-24 20:11:12,336 INFO [Listener at localhost/44145] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-24 20:11:12,336 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 20:11:12,336 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-24 20:11:12,336 INFO [Listener at localhost/44145] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-24 20:11:12,337 INFO [Listener at localhost/44145] ipc.NettyRpcServer(120): Bind to /172.31.14.131:38597 2023-07-24 20:11:12,337 INFO [Listener at localhost/44145] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-24 20:11:12,339 DEBUG [Listener at localhost/44145] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-24 20:11:12,339 INFO [Listener at localhost/44145] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:11:12,340 INFO [Listener at localhost/44145] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:11:12,341 INFO [Listener at localhost/44145] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:38597 connecting to ZooKeeper ensemble=127.0.0.1:62881 2023-07-24 20:11:12,345 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:385970x0, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-24 20:11:12,346 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): regionserver:385970x0, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-24 20:11:12,346 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): regionserver:385970x0, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 20:11:12,347 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:38597-0x10198f4f3ce001e connected 2023-07-24 20:11:12,347 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): regionserver:38597-0x10198f4f3ce001e, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-24 20:11:12,350 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38597 2023-07-24 20:11:12,352 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38597 2023-07-24 20:11:12,353 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38597 2023-07-24 20:11:12,353 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38597 2023-07-24 20:11:12,354 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38597 2023-07-24 20:11:12,356 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2023-07-24 20:11:12,356 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter) 2023-07-24 20:11:12,356 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'securityheaders' (class=org.apache.hadoop.hbase.http.SecurityHeadersFilter) 2023-07-24 20:11:12,356 INFO [Listener at localhost/44145] http.HttpServer(879): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context regionserver 2023-07-24 20:11:12,357 INFO [Listener at localhost/44145] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2023-07-24 20:11:12,357 INFO [Listener at localhost/44145] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2023-07-24 20:11:12,357 INFO [Listener at localhost/44145] http.HttpServer(783): ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint. 2023-07-24 20:11:12,357 INFO [Listener at localhost/44145] http.HttpServer(1146): Jetty bound to port 40735 2023-07-24 20:11:12,357 INFO [Listener at localhost/44145] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-24 20:11:12,363 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:11:12,363 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@3006b674{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/hadoop.log.dir/,AVAILABLE} 2023-07-24 20:11:12,363 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:11:12,363 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@3f748609{static,/static,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/static/,AVAILABLE} 2023-07-24 20:11:12,368 INFO [Listener at localhost/44145] webapp.StandardDescriptorProcessor(277): NO JSP Support for /, did not find org.apache.hbase.thirdparty.org.eclipse.jetty.jsp.JettyJspServlet 2023-07-24 20:11:12,369 INFO [Listener at localhost/44145] session.DefaultSessionIdManager(334): DefaultSessionIdManager workerName=node0 2023-07-24 20:11:12,369 INFO [Listener at localhost/44145] session.DefaultSessionIdManager(339): No SessionScavenger set, using defaults 2023-07-24 20:11:12,370 INFO [Listener at localhost/44145] session.HouseKeeper(132): node0 Scavenging every 660000ms 2023-07-24 20:11:12,375 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:11:12,376 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.w.WebAppContext@3e37ff10{regionserver,/,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver/,AVAILABLE}{file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver} 2023-07-24 20:11:12,377 INFO [Listener at localhost/44145] server.AbstractConnector(333): Started ServerConnector@34284301{HTTP/1.1, (http/1.1)}{0.0.0.0:40735} 2023-07-24 20:11:12,378 INFO [Listener at localhost/44145] server.Server(415): Started @40945ms 2023-07-24 20:11:12,390 INFO [Listener at localhost/44145] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-24 20:11:12,390 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 20:11:12,390 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-24 20:11:12,390 INFO [Listener at localhost/44145] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-24 20:11:12,390 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 20:11:12,390 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-24 20:11:12,390 INFO [Listener at localhost/44145] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-24 20:11:12,391 INFO [Listener at localhost/44145] ipc.NettyRpcServer(120): Bind to /172.31.14.131:34773 2023-07-24 20:11:12,391 INFO [Listener at localhost/44145] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-24 20:11:12,393 DEBUG [Listener at localhost/44145] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-24 20:11:12,393 INFO [Listener at localhost/44145] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:11:12,394 INFO [Listener at localhost/44145] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:11:12,395 INFO [Listener at localhost/44145] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:34773 connecting to ZooKeeper ensemble=127.0.0.1:62881 2023-07-24 20:11:12,399 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:347730x0, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-24 20:11:12,400 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): regionserver:347730x0, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-24 20:11:12,401 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): regionserver:347730x0, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 20:11:12,401 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): regionserver:347730x0, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-24 20:11:12,407 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:34773-0x10198f4f3ce001f connected 2023-07-24 20:11:12,407 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34773 2023-07-24 20:11:12,408 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34773 2023-07-24 20:11:12,410 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34773 2023-07-24 20:11:12,411 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34773 2023-07-24 20:11:12,413 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34773 2023-07-24 20:11:12,415 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2023-07-24 20:11:12,415 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter) 2023-07-24 20:11:12,415 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'securityheaders' (class=org.apache.hadoop.hbase.http.SecurityHeadersFilter) 2023-07-24 20:11:12,416 INFO [Listener at localhost/44145] http.HttpServer(879): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context regionserver 2023-07-24 20:11:12,416 INFO [Listener at localhost/44145] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2023-07-24 20:11:12,416 INFO [Listener at localhost/44145] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2023-07-24 20:11:12,416 INFO [Listener at localhost/44145] http.HttpServer(783): ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint. 2023-07-24 20:11:12,417 INFO [Listener at localhost/44145] http.HttpServer(1146): Jetty bound to port 34127 2023-07-24 20:11:12,417 INFO [Listener at localhost/44145] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-24 20:11:12,419 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:11:12,419 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@5405ec89{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/hadoop.log.dir/,AVAILABLE} 2023-07-24 20:11:12,419 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:11:12,419 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@409405c{static,/static,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/static/,AVAILABLE} 2023-07-24 20:11:12,425 INFO [Listener at localhost/44145] webapp.StandardDescriptorProcessor(277): NO JSP Support for /, did not find org.apache.hbase.thirdparty.org.eclipse.jetty.jsp.JettyJspServlet 2023-07-24 20:11:12,425 INFO [Listener at localhost/44145] session.DefaultSessionIdManager(334): DefaultSessionIdManager workerName=node0 2023-07-24 20:11:12,425 INFO [Listener at localhost/44145] session.DefaultSessionIdManager(339): No SessionScavenger set, using defaults 2023-07-24 20:11:12,426 INFO [Listener at localhost/44145] session.HouseKeeper(132): node0 Scavenging every 660000ms 2023-07-24 20:11:12,427 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:11:12,427 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.w.WebAppContext@514b4e4e{regionserver,/,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver/,AVAILABLE}{file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver} 2023-07-24 20:11:12,429 INFO [Listener at localhost/44145] server.AbstractConnector(333): Started ServerConnector@24f0b740{HTTP/1.1, (http/1.1)}{0.0.0.0:34127} 2023-07-24 20:11:12,429 INFO [Listener at localhost/44145] server.Server(415): Started @40996ms 2023-07-24 20:11:12,436 INFO [master/jenkins-hbase4:0:becomeActiveMaster] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-24 20:11:12,440 INFO [master/jenkins-hbase4:0:becomeActiveMaster] server.AbstractConnector(333): Started ServerConnector@13e4f5ac{HTTP/1.1, (http/1.1)}{0.0.0.0:44415} 2023-07-24 20:11:12,440 INFO [master/jenkins-hbase4:0:becomeActiveMaster] server.Server(415): Started @41007ms 2023-07-24 20:11:12,440 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,37489,1690229472216 2023-07-24 20:11:12,441 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-07-24 20:11:12,442 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,37489,1690229472216 2023-07-24 20:11:12,444 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:38597-0x10198f4f3ce001e, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-24 20:11:12,444 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-24 20:11:12,444 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34773-0x10198f4f3ce001f, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-24 20:11:12,444 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 20:11:12,445 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:46693-0x10198f4f3ce001d, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-24 20:11:12,448 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-24 20:11:12,449 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-24 20:11:12,449 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,37489,1690229472216 from backup master directory 2023-07-24 20:11:12,456 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,37489,1690229472216 2023-07-24 20:11:12,456 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-07-24 20:11:12,456 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-24 20:11:12,456 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,37489,1690229472216 2023-07-24 20:11:12,478 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:11:12,520 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x0bb4fccf to 127.0.0.1:62881 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-24 20:11:12,525 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@612091ab, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-24 20:11:12,525 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-24 20:11:12,526 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-24 20:11:12,526 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-24 20:11:12,533 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(288): Renamed hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/WALs/jenkins-hbase4.apache.org,33409,1690229459778 to hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/WALs/jenkins-hbase4.apache.org,33409,1690229459778-dead as it is dead 2023-07-24 20:11:12,533 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(86): Recover lease on dfs file hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/WALs/jenkins-hbase4.apache.org,33409,1690229459778-dead/jenkins-hbase4.apache.org%2C33409%2C1690229459778.1690229460221 2023-07-24 20:11:12,534 INFO [master/jenkins-hbase4:0:becomeActiveMaster] util.RecoverLeaseFSUtils(175): Recovered lease, attempt=0 on file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/WALs/jenkins-hbase4.apache.org,33409,1690229459778-dead/jenkins-hbase4.apache.org%2C33409%2C1690229459778.1690229460221 after 0ms 2023-07-24 20:11:12,535 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(300): Renamed hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/WALs/jenkins-hbase4.apache.org,33409,1690229459778-dead/jenkins-hbase4.apache.org%2C33409%2C1690229459778.1690229460221 to hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C33409%2C1690229459778.1690229460221 2023-07-24 20:11:12,535 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(302): Delete empty local region wal dir hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/WALs/jenkins-hbase4.apache.org,33409,1690229459778-dead 2023-07-24 20:11:12,535 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/WALs/jenkins-hbase4.apache.org,37489,1690229472216 2023-07-24 20:11:12,538 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37489%2C1690229472216, suffix=, logDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/WALs/jenkins-hbase4.apache.org,37489,1690229472216, archiveDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/oldWALs, maxLogs=10 2023-07-24 20:11:12,555 DEBUG [RS-EventLoopGroup-16-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK] 2023-07-24 20:11:12,555 DEBUG [RS-EventLoopGroup-16-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK] 2023-07-24 20:11:12,555 DEBUG [RS-EventLoopGroup-16-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK] 2023-07-24 20:11:12,562 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/WALs/jenkins-hbase4.apache.org,37489,1690229472216/jenkins-hbase4.apache.org%2C37489%2C1690229472216.1690229472538 2023-07-24 20:11:12,562 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK], DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK], DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK]] 2023-07-24 20:11:12,562 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-24 20:11:12,562 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:11:12,562 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-24 20:11:12,562 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-24 20:11:12,564 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-24 20:11:12,565 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-24 20:11:12,566 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-24 20:11:12,576 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/273705b3a8934e9f89526eacd88c3946 2023-07-24 20:11:12,581 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(539): loaded hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/b21deb618bb94cbcaa2b660c8c643e86 2023-07-24 20:11:12,581 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:11:12,582 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5179): Found 1 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals 2023-07-24 20:11:12,582 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5276): Replaying edits from hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C33409%2C1690229459778.1690229460221 2023-07-24 20:11:12,587 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5464): Applied 0, skipped 127, firstSequenceIdInLog=800, maxSequenceIdInLog=911, path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C33409%2C1690229459778.1690229460221 2023-07-24 20:11:12,592 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5086): Deleted recovered.edits file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.wals/jenkins-hbase4.apache.org%2C33409%2C1690229459778.1690229460221 2023-07-24 20:11:12,596 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-24 20:11:12,599 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/911.seqid, newMaxSeqId=911, maxSeqId=798 2023-07-24 20:11:12,600 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=912; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=12046116160, jitterRate=0.12188199162483215}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 20:11:12,600 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-24 20:11:12,600 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-24 20:11:12,602 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-07-24 20:11:12,602 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-07-24 20:11:12,602 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 20:11:12,602 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 20:11:12,624 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 20:11:12,624 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 20:11:12,625 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=6, state=SUCCESS; CreateTableProcedure table=hbase:rsgroup 2023-07-24 20:11:12,625 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=10, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 20:11:12,625 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=11, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 20:11:12,625 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, REOPEN/MOVE 2023-07-24 20:11:12,625 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=15, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,34371,1690229440021, splitWal=true, meta=false 2023-07-24 20:11:12,626 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=16, state=SUCCESS; ModifyNamespaceProcedure, namespace=default 2023-07-24 20:11:12,626 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=17, state=SUCCESS; CreateTableProcedure table=Group_testCreateAndAssign 2023-07-24 20:11:12,626 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=20, state=SUCCESS; DisableTableProcedure table=Group_testCreateAndAssign 2023-07-24 20:11:12,627 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=23, state=SUCCESS; DeleteTableProcedure table=Group_testCreateAndAssign 2023-07-24 20:11:12,627 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; CreateTableProcedure table=Group_testCreateMultiRegion 2023-07-24 20:11:12,628 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=45, state=SUCCESS; DisableTableProcedure table=Group_testCreateMultiRegion 2023-07-24 20:11:12,628 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=66, state=SUCCESS; DeleteTableProcedure table=Group_testCreateMultiRegion 2023-07-24 20:11:12,628 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=67, state=SUCCESS; CreateNamespaceProcedure, namespace=Group_foo 2023-07-24 20:11:12,628 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; CreateTableProcedure table=Group_foo:Group_testCreateAndAssign 2023-07-24 20:11:12,629 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=71, state=SUCCESS; DisableTableProcedure table=Group_foo:Group_testCreateAndAssign 2023-07-24 20:11:12,629 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=74, state=SUCCESS; DeleteTableProcedure table=Group_foo:Group_testCreateAndAssign 2023-07-24 20:11:12,629 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=75, state=SUCCESS; DeleteNamespaceProcedure, namespace=Group_foo 2023-07-24 20:11:12,629 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=76, state=SUCCESS; CreateTableProcedure table=Group_testCreateAndDrop 2023-07-24 20:11:12,629 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=Group_testCreateAndDrop 2023-07-24 20:11:12,629 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=82, state=SUCCESS; DeleteTableProcedure table=Group_testCreateAndDrop 2023-07-24 20:11:12,630 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=83, state=SUCCESS; CreateTableProcedure table=Group_testCloneSnapshot 2023-07-24 20:11:12,630 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=86, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=Group_testCloneSnapshot, type=EXCLUSIVE 2023-07-24 20:11:12,630 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=87, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=Group_testCloneSnapshot, type=SHARED 2023-07-24 20:11:12,631 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; CloneSnapshotProcedure (table=Group_testCloneSnapshot_clone snapshot=name: "Group_testCloneSnapshot_snap" table: "Group_testCloneSnapshot" creation_time: 1690229452851 type: FLUSH version: 2 ttl: 0 ) 2023-07-24 20:11:12,631 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=91, state=SUCCESS; DisableTableProcedure table=Group_testCloneSnapshot 2023-07-24 20:11:12,631 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=94, state=SUCCESS; DeleteTableProcedure table=Group_testCloneSnapshot 2023-07-24 20:11:12,633 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=95, state=SUCCESS; DisableTableProcedure table=Group_testCloneSnapshot_clone 2023-07-24 20:11:12,633 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=98, state=SUCCESS; DeleteTableProcedure table=Group_testCloneSnapshot_clone 2023-07-24 20:11:12,634 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=99, state=SUCCESS; CreateNamespaceProcedure, namespace=Group_ns 2023-07-24 20:11:12,634 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=100, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-create-table:org.apache.hadoop.hbase.HBaseIOException: No online servers in the rsgroup appInfo which table Group_ns:testCreateWhenRsgroupNoOnlineServers belongs to; CreateTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-24 20:11:12,635 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; CreateTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-24 20:11:12,635 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=104, state=SUCCESS; DisableTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-24 20:11:12,635 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=107, state=SUCCESS; DeleteTableProcedure table=Group_ns:testCreateWhenRsgroupNoOnlineServers 2023-07-24 20:11:12,635 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteNamespaceProcedure, namespace=Group_ns 2023-07-24 20:11:12,636 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,39791,1690229440084, splitWal=true, meta=false 2023-07-24 20:11:12,636 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=110, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,45209,1690229445078, splitWal=true, meta=false 2023-07-24 20:11:12,636 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=111, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,34483,1690229439858, splitWal=true, meta=false 2023-07-24 20:11:12,636 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=112, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,34917,1690229443609, splitWal=true, meta=true 2023-07-24 20:11:12,636 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(411): Completed pid=119, state=SUCCESS; CreateTableProcedure table=hbase:quota 2023-07-24 20:11:12,637 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 34 msec 2023-07-24 20:11:12,637 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-24 20:11:12,647 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 20:11:12,648 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,46705,1690229459880, table=hbase:meta, region=1588230740 2023-07-24 20:11:12,650 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 3 possibly 'live' servers, and 0 'splitting'. 2023-07-24 20:11:12,656 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,45019,1690229459931 already deleted, retry=false 2023-07-24 20:11:12,657 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,45019,1690229459931 on jenkins-hbase4.apache.org,37489,1690229472216 2023-07-24 20:11:12,658 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=122, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,45019,1690229459931, splitWal=true, meta=false 2023-07-24 20:11:12,659 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=122 for jenkins-hbase4.apache.org,45019,1690229459931 (carryingMeta=false) jenkins-hbase4.apache.org,45019,1690229459931/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@3dc119db[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-24 20:11:12,660 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,46065,1690229460063 already deleted, retry=false 2023-07-24 20:11:12,660 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,46065,1690229460063 on jenkins-hbase4.apache.org,37489,1690229472216 2023-07-24 20:11:12,662 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=123, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,46065,1690229460063, splitWal=true, meta=false 2023-07-24 20:11:12,663 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=123 for jenkins-hbase4.apache.org,46065,1690229460063 (carryingMeta=false) jenkins-hbase4.apache.org,46065,1690229460063/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@591a3694[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-24 20:11:12,664 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,46705,1690229459880 already deleted, retry=false 2023-07-24 20:11:12,664 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,46705,1690229459880 on jenkins-hbase4.apache.org,37489,1690229472216 2023-07-24 20:11:12,667 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,46705,1690229459880, splitWal=true, meta=true 2023-07-24 20:11:12,667 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=124 for jenkins-hbase4.apache.org,46705,1690229459880 (carryingMeta=true) jenkins-hbase4.apache.org,46705,1690229459880/CRASHED/regionCount=1/lock=java.util.concurrent.locks.ReentrantReadWriteLock@2b4b8c41[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-24 20:11:12,668 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/balancer 2023-07-24 20:11:12,668 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-24 20:11:12,669 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-07-24 20:11:12,670 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-07-24 20:11:12,670 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-07-24 20:11:12,671 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-07-24 20:11:12,673 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34773-0x10198f4f3ce001f, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-24 20:11:12,673 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:46693-0x10198f4f3ce001d, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-24 20:11:12,673 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-24 20:11:12,673 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 20:11:12,673 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:38597-0x10198f4f3ce001e, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-24 20:11:12,674 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,37489,1690229472216, sessionid=0x10198f4f3ce001c, setting cluster-up flag (Was=false) 2023-07-24 20:11:12,679 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-07-24 20:11:12,680 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,37489,1690229472216 2023-07-24 20:11:12,682 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-07-24 20:11:12,685 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,37489,1690229472216 2023-07-24 20:11:12,686 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/.hbase-snapshot/.tmp 2023-07-24 20:11:12,687 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2930): Registered master coprocessor service: service=RSGroupAdminService 2023-07-24 20:11:12,687 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] rsgroup.RSGroupInfoManagerImpl(537): Refreshing in Offline mode. 2023-07-24 20:11:12,688 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] rsgroup.RSGroupInfoManagerImpl(511): Read ZK GroupInfo count:2 2023-07-24 20:11:12,689 INFO [master/jenkins-hbase4:0:becomeActiveMaster] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint loaded, priority=536870911. 2023-07-24 20:11:12,689 INFO [master/jenkins-hbase4:0:becomeActiveMaster] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase$CPMasterObserver loaded, priority=536870912. 2023-07-24 20:11:12,690 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,37489,1690229472216] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(787): Updating default servers. 2023-07-24 20:11:12,692 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,37489,1690229472216] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-24 20:11:12,693 WARN [RS-EventLoopGroup-16-3] ipc.NettyRpcConnection$2(294): Exception encountered while connecting to the server jenkins-hbase4.apache.org/172.31.14.131:46705 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:46705 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:155) at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:128) at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-07-24 20:11:12,695 DEBUG [RS-EventLoopGroup-16-3] ipc.FailedServers(52): Added failed server with address jenkins-hbase4.apache.org/172.31.14.131:46705 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:46705 2023-07-24 20:11:12,705 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-24 20:11:12,705 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-24 20:11:12,705 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-24 20:11:12,705 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-24 20:11:12,706 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-24 20:11:12,706 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-24 20:11:12,706 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-24 20:11:12,706 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-07-24 20:11:12,706 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-07-24 20:11:12,706 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:12,706 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-24 20:11:12,706 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:12,714 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690229502714 2023-07-24 20:11:12,715 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-24 20:11:12,715 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-24 20:11:12,715 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-24 20:11:12,716 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-24 20:11:12,716 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-24 20:11:12,716 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-24 20:11:12,718 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:12,727 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-24 20:11:12,727 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-24 20:11:12,728 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-24 20:11:12,728 DEBUG [PEWorker-1] master.DeadServer(103): Processing jenkins-hbase4.apache.org,46705,1690229459880; numProcessing=1 2023-07-24 20:11:12,728 INFO [PEWorker-1] procedure.ServerCrashProcedure(161): Start pid=124, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,46705,1690229459880, splitWal=true, meta=true 2023-07-24 20:11:12,732 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-24 20:11:12,732 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-24 20:11:12,732 DEBUG [PEWorker-2] master.DeadServer(103): Processing jenkins-hbase4.apache.org,45019,1690229459931; numProcessing=2 2023-07-24 20:11:12,732 INFO [PEWorker-2] procedure.ServerCrashProcedure(161): Start pid=122, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,45019,1690229459931, splitWal=true, meta=false 2023-07-24 20:11:12,735 INFO [PEWorker-1] procedure.ServerCrashProcedure(300): Splitting WALs pid=124, state=RUNNABLE:SERVER_CRASH_SPLIT_META_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,46705,1690229459880, splitWal=true, meta=true, isMeta: true 2023-07-24 20:11:12,735 DEBUG [PEWorker-4] master.DeadServer(103): Processing jenkins-hbase4.apache.org,46065,1690229460063; numProcessing=3 2023-07-24 20:11:12,735 INFO [PEWorker-4] procedure.ServerCrashProcedure(161): Start pid=123, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,46065,1690229460063, splitWal=true, meta=false 2023-07-24 20:11:12,736 INFO [RS:1;jenkins-hbase4:38597] regionserver.HRegionServer(951): ClusterId : dc54b028-3544-4133-8f0e-d550a19bc790 2023-07-24 20:11:12,736 DEBUG [RS:1;jenkins-hbase4:38597] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-24 20:11:12,736 INFO [RS:0;jenkins-hbase4:46693] regionserver.HRegionServer(951): ClusterId : dc54b028-3544-4133-8f0e-d550a19bc790 2023-07-24 20:11:12,736 DEBUG [RS:0;jenkins-hbase4:46693] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-24 20:11:12,738 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1690229472734,5,FailOnTimeoutGroup] 2023-07-24 20:11:12,739 DEBUG [RS:1;jenkins-hbase4:38597] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-24 20:11:12,739 DEBUG [RS:1;jenkins-hbase4:38597] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-24 20:11:12,739 DEBUG [RS:0;jenkins-hbase4:46693] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-24 20:11:12,739 DEBUG [RS:0;jenkins-hbase4:46693] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-24 20:11:12,741 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1690229472739,5,FailOnTimeoutGroup] 2023-07-24 20:11:12,741 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:12,741 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-24 20:11:12,741 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:12,741 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:12,741 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1690229472741, completionTime=-1 2023-07-24 20:11:12,741 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(766): The value of 'hbase.master.wait.on.regionservers.maxtostart' (-1) is set less than 'hbase.master.wait.on.regionservers.mintostart' (1), ignoring. 2023-07-24 20:11:12,741 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=0; waited=0ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-24 20:11:12,742 DEBUG [PEWorker-1] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,46705,1690229459880-splitting 2023-07-24 20:11:12,742 DEBUG [RS:1;jenkins-hbase4:38597] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-24 20:11:12,742 INFO [RS:2;jenkins-hbase4:34773] regionserver.HRegionServer(951): ClusterId : dc54b028-3544-4133-8f0e-d550a19bc790 2023-07-24 20:11:12,743 DEBUG [RS:2;jenkins-hbase4:34773] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-24 20:11:12,744 DEBUG [RS:0;jenkins-hbase4:46693] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-24 20:11:12,744 DEBUG [RS:1;jenkins-hbase4:38597] zookeeper.ReadOnlyZKClient(139): Connect 0x47c01838 to 127.0.0.1:62881 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-24 20:11:12,745 DEBUG [RS:0;jenkins-hbase4:46693] zookeeper.ReadOnlyZKClient(139): Connect 0x1c312feb to 127.0.0.1:62881 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-24 20:11:12,746 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,46705,1690229459880-splitting dir is empty, no logs to split. 2023-07-24 20:11:12,746 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,46705,1690229459880 WAL count=0, meta=true 2023-07-24 20:11:12,746 DEBUG [RS:2;jenkins-hbase4:34773] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-24 20:11:12,746 DEBUG [RS:2;jenkins-hbase4:34773] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-24 20:11:12,748 DEBUG [RS:2;jenkins-hbase4:34773] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-24 20:11:12,756 DEBUG [RS:2;jenkins-hbase4:34773] zookeeper.ReadOnlyZKClient(139): Connect 0x68001b57 to 127.0.0.1:62881 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-24 20:11:12,756 INFO [PEWorker-1] master.SplitLogManager(171): hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,46705,1690229459880-splitting dir is empty, no logs to split. 2023-07-24 20:11:12,756 INFO [PEWorker-1] master.SplitWALManager(106): jenkins-hbase4.apache.org,46705,1690229459880 WAL count=0, meta=true 2023-07-24 20:11:12,756 DEBUG [PEWorker-1] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,46705,1690229459880 WAL splitting is done? wals=0, meta=true 2023-07-24 20:11:12,757 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=125, ppid=124, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-24 20:11:12,759 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=125, ppid=124, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-24 20:11:12,761 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=125, ppid=124, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-24 20:11:12,768 DEBUG [RS:1;jenkins-hbase4:38597] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4907eeb0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-24 20:11:12,768 DEBUG [RS:1;jenkins-hbase4:38597] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4d387348, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-24 20:11:12,768 DEBUG [RS:0;jenkins-hbase4:46693] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@12f6d21d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-24 20:11:12,768 DEBUG [RS:0;jenkins-hbase4:46693] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@16c9449b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-24 20:11:12,769 DEBUG [RS:2;jenkins-hbase4:34773] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@66c7e827, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-24 20:11:12,769 DEBUG [RS:2;jenkins-hbase4:34773] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@62f87027, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-24 20:11:12,776 DEBUG [RS:1;jenkins-hbase4:38597] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:38597 2023-07-24 20:11:12,777 INFO [RS:1;jenkins-hbase4:38597] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-24 20:11:12,777 INFO [RS:1;jenkins-hbase4:38597] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-24 20:11:12,777 DEBUG [RS:1;jenkins-hbase4:38597] regionserver.HRegionServer(1022): About to register with Master. 2023-07-24 20:11:12,777 INFO [RS:1;jenkins-hbase4:38597] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,37489,1690229472216 with isa=jenkins-hbase4.apache.org/172.31.14.131:38597, startcode=1690229472335 2023-07-24 20:11:12,777 DEBUG [RS:2;jenkins-hbase4:34773] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:34773 2023-07-24 20:11:12,777 DEBUG [RS:1;jenkins-hbase4:38597] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-24 20:11:12,778 INFO [RS:2;jenkins-hbase4:34773] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-24 20:11:12,778 INFO [RS:2;jenkins-hbase4:34773] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-24 20:11:12,778 DEBUG [RS:2;jenkins-hbase4:34773] regionserver.HRegionServer(1022): About to register with Master. 2023-07-24 20:11:12,778 INFO [RS:2;jenkins-hbase4:34773] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,37489,1690229472216 with isa=jenkins-hbase4.apache.org/172.31.14.131:34773, startcode=1690229472389 2023-07-24 20:11:12,778 DEBUG [RS:2;jenkins-hbase4:34773] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-24 20:11:12,779 INFO [RS-EventLoopGroup-13-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43049, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.9 (auth:SIMPLE), service=RegionServerStatusService 2023-07-24 20:11:12,780 INFO [RS-EventLoopGroup-13-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54877, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.10 (auth:SIMPLE), service=RegionServerStatusService 2023-07-24 20:11:12,781 DEBUG [RS:0;jenkins-hbase4:46693] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:46693 2023-07-24 20:11:12,781 INFO [RS:0;jenkins-hbase4:46693] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-24 20:11:12,781 INFO [RS:0;jenkins-hbase4:46693] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-24 20:11:12,781 DEBUG [RS:0;jenkins-hbase4:46693] regionserver.HRegionServer(1022): About to register with Master. 2023-07-24 20:11:12,783 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=37489] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,38597,1690229472335 2023-07-24 20:11:12,783 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,37489,1690229472216] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(787): Updating default servers. 2023-07-24 20:11:12,784 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=37489] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,34773,1690229472389 2023-07-24 20:11:12,785 DEBUG [RS:2;jenkins-hbase4:34773] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4 2023-07-24 20:11:12,785 DEBUG [RS:2;jenkins-hbase4:34773] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:37609 2023-07-24 20:11:12,785 DEBUG [RS:2;jenkins-hbase4:34773] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=43499 2023-07-24 20:11:12,785 INFO [RS:0;jenkins-hbase4:46693] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,37489,1690229472216 with isa=jenkins-hbase4.apache.org/172.31.14.131:46693, startcode=1690229472273 2023-07-24 20:11:12,785 DEBUG [RS:0;jenkins-hbase4:46693] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-24 20:11:12,786 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:11:12,787 INFO [RS-EventLoopGroup-13-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36573, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.8 (auth:SIMPLE), service=RegionServerStatusService 2023-07-24 20:11:12,787 DEBUG [RS:2;jenkins-hbase4:34773] zookeeper.ZKUtil(162): regionserver:34773-0x10198f4f3ce001f, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34773,1690229472389 2023-07-24 20:11:12,787 WARN [RS:2;jenkins-hbase4:34773] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-24 20:11:12,787 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=37489] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,46693,1690229472273 2023-07-24 20:11:12,787 INFO [RS:2;jenkins-hbase4:34773] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-24 20:11:12,787 DEBUG [RS:2;jenkins-hbase4:34773] regionserver.HRegionServer(1948): logDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,34773,1690229472389 2023-07-24 20:11:12,787 DEBUG [RS:0;jenkins-hbase4:46693] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4 2023-07-24 20:11:12,787 DEBUG [RS:0;jenkins-hbase4:46693] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:37609 2023-07-24 20:11:12,787 DEBUG [RS:0;jenkins-hbase4:46693] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=43499 2023-07-24 20:11:12,791 DEBUG [RS:1;jenkins-hbase4:38597] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4 2023-07-24 20:11:12,791 DEBUG [RS:0;jenkins-hbase4:46693] zookeeper.ZKUtil(162): regionserver:46693-0x10198f4f3ce001d, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46693,1690229472273 2023-07-24 20:11:12,791 DEBUG [RS:1;jenkins-hbase4:38597] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:37609 2023-07-24 20:11:12,791 WARN [RS:0;jenkins-hbase4:46693] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-24 20:11:12,791 DEBUG [RS:1;jenkins-hbase4:38597] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=43499 2023-07-24 20:11:12,791 INFO [RS:0;jenkins-hbase4:46693] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-24 20:11:12,791 DEBUG [RS:0;jenkins-hbase4:46693] regionserver.HRegionServer(1948): logDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,46693,1690229472273 2023-07-24 20:11:12,791 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=3; waited=50ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 2023-07-24 20:11:12,792 DEBUG [RS:1;jenkins-hbase4:38597] zookeeper.ZKUtil(162): regionserver:38597-0x10198f4f3ce001e, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38597,1690229472335 2023-07-24 20:11:12,792 WARN [RS:1;jenkins-hbase4:38597] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-24 20:11:12,792 INFO [RS:1;jenkins-hbase4:38597] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-24 20:11:12,792 DEBUG [RS:1;jenkins-hbase4:38597] regionserver.HRegionServer(1948): logDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,38597,1690229472335 2023-07-24 20:11:12,795 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,37489,1690229472216] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(792): Updated with servers: 1 2023-07-24 20:11:12,795 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,37489,1690229472216] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(787): Updating default servers. 2023-07-24 20:11:12,796 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,37489,1690229472216] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(792): Updated with servers: 3 2023-07-24 20:11:12,797 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,38597,1690229472335] 2023-07-24 20:11:12,797 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,34773,1690229472389] 2023-07-24 20:11:12,797 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,46693,1690229472273] 2023-07-24 20:11:12,801 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,37489,1690229472216] ipc.AbstractRpcClient(347): Not trying to connect to jenkins-hbase4.apache.org/172.31.14.131:46705 this server is in the failed servers list 2023-07-24 20:11:12,803 DEBUG [RS:2;jenkins-hbase4:34773] zookeeper.ZKUtil(162): regionserver:34773-0x10198f4f3ce001f, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34773,1690229472389 2023-07-24 20:11:12,803 DEBUG [RS:1;jenkins-hbase4:38597] zookeeper.ZKUtil(162): regionserver:38597-0x10198f4f3ce001e, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34773,1690229472389 2023-07-24 20:11:12,803 DEBUG [RS:0;jenkins-hbase4:46693] zookeeper.ZKUtil(162): regionserver:46693-0x10198f4f3ce001d, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34773,1690229472389 2023-07-24 20:11:12,803 DEBUG [RS:2;jenkins-hbase4:34773] zookeeper.ZKUtil(162): regionserver:34773-0x10198f4f3ce001f, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38597,1690229472335 2023-07-24 20:11:12,803 DEBUG [RS:1;jenkins-hbase4:38597] zookeeper.ZKUtil(162): regionserver:38597-0x10198f4f3ce001e, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38597,1690229472335 2023-07-24 20:11:12,803 DEBUG [RS:0;jenkins-hbase4:46693] zookeeper.ZKUtil(162): regionserver:46693-0x10198f4f3ce001d, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38597,1690229472335 2023-07-24 20:11:12,803 DEBUG [RS:2;jenkins-hbase4:34773] zookeeper.ZKUtil(162): regionserver:34773-0x10198f4f3ce001f, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46693,1690229472273 2023-07-24 20:11:12,804 DEBUG [RS:1;jenkins-hbase4:38597] zookeeper.ZKUtil(162): regionserver:38597-0x10198f4f3ce001e, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46693,1690229472273 2023-07-24 20:11:12,804 DEBUG [RS:0;jenkins-hbase4:46693] zookeeper.ZKUtil(162): regionserver:46693-0x10198f4f3ce001d, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46693,1690229472273 2023-07-24 20:11:12,804 DEBUG [RS:2;jenkins-hbase4:34773] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-24 20:11:12,804 DEBUG [RS:1;jenkins-hbase4:38597] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-24 20:11:12,805 DEBUG [RS:0;jenkins-hbase4:46693] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-24 20:11:12,805 INFO [RS:1;jenkins-hbase4:38597] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-24 20:11:12,805 INFO [RS:2;jenkins-hbase4:34773] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-24 20:11:12,805 INFO [RS:0;jenkins-hbase4:46693] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-24 20:11:12,806 INFO [RS:1;jenkins-hbase4:38597] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-24 20:11:12,808 INFO [RS:1;jenkins-hbase4:38597] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-24 20:11:12,808 INFO [RS:1;jenkins-hbase4:38597] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:12,808 INFO [RS:1;jenkins-hbase4:38597] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-24 20:11:12,815 INFO [RS:2;jenkins-hbase4:34773] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-24 20:11:12,816 INFO [RS:0;jenkins-hbase4:46693] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-24 20:11:12,818 INFO [RS:2;jenkins-hbase4:34773] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-24 20:11:12,818 INFO [RS:2;jenkins-hbase4:34773] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:12,819 INFO [RS:1;jenkins-hbase4:38597] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:12,819 INFO [RS:2;jenkins-hbase4:34773] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-24 20:11:12,819 INFO [RS:0;jenkins-hbase4:46693] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-24 20:11:12,819 DEBUG [RS:1;jenkins-hbase4:38597] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:12,820 INFO [RS:0;jenkins-hbase4:46693] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:12,820 DEBUG [RS:1;jenkins-hbase4:38597] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:12,820 DEBUG [RS:1;jenkins-hbase4:38597] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:12,820 DEBUG [RS:1;jenkins-hbase4:38597] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:12,820 DEBUG [RS:1;jenkins-hbase4:38597] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:12,820 DEBUG [RS:1;jenkins-hbase4:38597] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-24 20:11:12,820 INFO [RS:0;jenkins-hbase4:46693] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-24 20:11:12,821 DEBUG [RS:1;jenkins-hbase4:38597] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:12,821 DEBUG [RS:1;jenkins-hbase4:38597] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:12,821 INFO [RS:2;jenkins-hbase4:34773] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:12,821 DEBUG [RS:1;jenkins-hbase4:38597] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:12,821 DEBUG [RS:1;jenkins-hbase4:38597] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:12,821 DEBUG [RS:2;jenkins-hbase4:34773] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:12,822 DEBUG [RS:2;jenkins-hbase4:34773] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:12,822 DEBUG [RS:2;jenkins-hbase4:34773] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:12,822 INFO [RS:0;jenkins-hbase4:46693] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:12,822 DEBUG [RS:2;jenkins-hbase4:34773] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:12,822 INFO [RS:1;jenkins-hbase4:38597] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:12,823 DEBUG [RS:0;jenkins-hbase4:46693] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:12,823 INFO [RS:1;jenkins-hbase4:38597] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:12,823 DEBUG [RS:0;jenkins-hbase4:46693] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:12,823 DEBUG [RS:2;jenkins-hbase4:34773] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:12,823 DEBUG [RS:0;jenkins-hbase4:46693] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:12,823 DEBUG [RS:2;jenkins-hbase4:34773] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-24 20:11:12,823 INFO [RS:1;jenkins-hbase4:38597] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:12,823 DEBUG [RS:2;jenkins-hbase4:34773] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:12,823 DEBUG [RS:0;jenkins-hbase4:46693] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:12,823 DEBUG [RS:2;jenkins-hbase4:34773] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:12,823 DEBUG [RS:0;jenkins-hbase4:46693] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:12,823 DEBUG [RS:0;jenkins-hbase4:46693] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-24 20:11:12,823 DEBUG [RS:2;jenkins-hbase4:34773] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:12,823 DEBUG [RS:0;jenkins-hbase4:46693] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:12,824 DEBUG [RS:2;jenkins-hbase4:34773] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:12,824 DEBUG [RS:0;jenkins-hbase4:46693] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:12,824 DEBUG [RS:0;jenkins-hbase4:46693] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:12,824 DEBUG [RS:0;jenkins-hbase4:46693] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:12,830 INFO [RS:2;jenkins-hbase4:34773] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:12,830 INFO [RS:2;jenkins-hbase4:34773] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:12,830 INFO [RS:2;jenkins-hbase4:34773] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:12,833 INFO [RS:0;jenkins-hbase4:46693] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:12,833 INFO [RS:0;jenkins-hbase4:46693] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:12,833 INFO [RS:0;jenkins-hbase4:46693] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:12,840 INFO [RS:1;jenkins-hbase4:38597] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-24 20:11:12,840 INFO [RS:1;jenkins-hbase4:38597] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38597,1690229472335-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:12,852 INFO [RS:2;jenkins-hbase4:34773] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-24 20:11:12,852 INFO [RS:2;jenkins-hbase4:34773] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,34773,1690229472389-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:12,854 INFO [RS:0;jenkins-hbase4:46693] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-24 20:11:12,855 INFO [RS:0;jenkins-hbase4:46693] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,46693,1690229472273-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:12,863 INFO [RS:1;jenkins-hbase4:38597] regionserver.Replication(203): jenkins-hbase4.apache.org,38597,1690229472335 started 2023-07-24 20:11:12,864 INFO [RS:1;jenkins-hbase4:38597] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,38597,1690229472335, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:38597, sessionid=0x10198f4f3ce001e 2023-07-24 20:11:12,864 DEBUG [RS:1;jenkins-hbase4:38597] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-24 20:11:12,864 DEBUG [RS:1;jenkins-hbase4:38597] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,38597,1690229472335 2023-07-24 20:11:12,864 DEBUG [RS:1;jenkins-hbase4:38597] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,38597,1690229472335' 2023-07-24 20:11:12,864 DEBUG [RS:1;jenkins-hbase4:38597] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 20:11:12,864 DEBUG [RS:1;jenkins-hbase4:38597] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 20:11:12,865 DEBUG [RS:1;jenkins-hbase4:38597] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-24 20:11:12,865 DEBUG [RS:1;jenkins-hbase4:38597] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-24 20:11:12,865 DEBUG [RS:1;jenkins-hbase4:38597] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,38597,1690229472335 2023-07-24 20:11:12,865 DEBUG [RS:1;jenkins-hbase4:38597] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,38597,1690229472335' 2023-07-24 20:11:12,865 DEBUG [RS:1;jenkins-hbase4:38597] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-24 20:11:12,865 INFO [RS:2;jenkins-hbase4:34773] regionserver.Replication(203): jenkins-hbase4.apache.org,34773,1690229472389 started 2023-07-24 20:11:12,865 INFO [RS:2;jenkins-hbase4:34773] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,34773,1690229472389, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:34773, sessionid=0x10198f4f3ce001f 2023-07-24 20:11:12,865 DEBUG [RS:2;jenkins-hbase4:34773] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-24 20:11:12,865 DEBUG [RS:1;jenkins-hbase4:38597] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-24 20:11:12,865 DEBUG [RS:2;jenkins-hbase4:34773] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,34773,1690229472389 2023-07-24 20:11:12,866 DEBUG [RS:2;jenkins-hbase4:34773] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34773,1690229472389' 2023-07-24 20:11:12,866 DEBUG [RS:2;jenkins-hbase4:34773] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 20:11:12,866 DEBUG [RS:1;jenkins-hbase4:38597] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-24 20:11:12,866 INFO [RS:1;jenkins-hbase4:38597] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-24 20:11:12,866 INFO [RS:1;jenkins-hbase4:38597] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-24 20:11:12,866 DEBUG [RS:2;jenkins-hbase4:34773] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 20:11:12,868 DEBUG [RS:2;jenkins-hbase4:34773] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-24 20:11:12,869 DEBUG [RS:2;jenkins-hbase4:34773] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-24 20:11:12,869 DEBUG [RS:2;jenkins-hbase4:34773] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,34773,1690229472389 2023-07-24 20:11:12,869 DEBUG [RS:2;jenkins-hbase4:34773] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,34773,1690229472389' 2023-07-24 20:11:12,869 DEBUG [RS:2;jenkins-hbase4:34773] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-24 20:11:12,869 DEBUG [RS:2;jenkins-hbase4:34773] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-24 20:11:12,869 DEBUG [RS:2;jenkins-hbase4:34773] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-24 20:11:12,869 INFO [RS:2;jenkins-hbase4:34773] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-24 20:11:12,869 INFO [RS:2;jenkins-hbase4:34773] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-24 20:11:12,872 INFO [RS:0;jenkins-hbase4:46693] regionserver.Replication(203): jenkins-hbase4.apache.org,46693,1690229472273 started 2023-07-24 20:11:12,873 INFO [RS:0;jenkins-hbase4:46693] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,46693,1690229472273, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:46693, sessionid=0x10198f4f3ce001d 2023-07-24 20:11:12,873 DEBUG [RS:0;jenkins-hbase4:46693] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-24 20:11:12,873 DEBUG [RS:0;jenkins-hbase4:46693] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,46693,1690229472273 2023-07-24 20:11:12,873 DEBUG [RS:0;jenkins-hbase4:46693] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46693,1690229472273' 2023-07-24 20:11:12,873 DEBUG [RS:0;jenkins-hbase4:46693] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 20:11:12,873 DEBUG [RS:0;jenkins-hbase4:46693] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 20:11:12,873 DEBUG [RS:0;jenkins-hbase4:46693] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-24 20:11:12,873 DEBUG [RS:0;jenkins-hbase4:46693] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-24 20:11:12,873 DEBUG [RS:0;jenkins-hbase4:46693] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,46693,1690229472273 2023-07-24 20:11:12,873 DEBUG [RS:0;jenkins-hbase4:46693] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,46693,1690229472273' 2023-07-24 20:11:12,874 DEBUG [RS:0;jenkins-hbase4:46693] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-24 20:11:12,874 DEBUG [RS:0;jenkins-hbase4:46693] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-24 20:11:12,874 DEBUG [RS:0;jenkins-hbase4:46693] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-24 20:11:12,874 INFO [RS:0;jenkins-hbase4:46693] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-24 20:11:12,874 INFO [RS:0;jenkins-hbase4:46693] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-24 20:11:12,916 DEBUG [jenkins-hbase4:37489] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-07-24 20:11:12,916 DEBUG [jenkins-hbase4:37489] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-24 20:11:12,916 DEBUG [jenkins-hbase4:37489] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 20:11:12,917 DEBUG [jenkins-hbase4:37489] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 20:11:12,917 DEBUG [jenkins-hbase4:37489] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 20:11:12,917 DEBUG [jenkins-hbase4:37489] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 20:11:12,918 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,46693,1690229472273, state=OPENING 2023-07-24 20:11:12,920 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-07-24 20:11:12,920 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-24 20:11:12,920 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,46693,1690229472273}] 2023-07-24 20:11:12,968 INFO [RS:1;jenkins-hbase4:38597] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38597%2C1690229472335, suffix=, logDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,38597,1690229472335, archiveDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/oldWALs, maxLogs=32 2023-07-24 20:11:12,977 INFO [RS:0;jenkins-hbase4:46693] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46693%2C1690229472273, suffix=, logDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,46693,1690229472273, archiveDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/oldWALs, maxLogs=32 2023-07-24 20:11:12,977 INFO [RS:2;jenkins-hbase4:34773] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C34773%2C1690229472389, suffix=, logDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,34773,1690229472389, archiveDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/oldWALs, maxLogs=32 2023-07-24 20:11:12,991 DEBUG [RS-EventLoopGroup-16-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK] 2023-07-24 20:11:12,991 DEBUG [RS-EventLoopGroup-16-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK] 2023-07-24 20:11:12,992 DEBUG [RS-EventLoopGroup-16-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK] 2023-07-24 20:11:12,995 INFO [RS:1;jenkins-hbase4:38597] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,38597,1690229472335/jenkins-hbase4.apache.org%2C38597%2C1690229472335.1690229472968 2023-07-24 20:11:13,002 DEBUG [RS:1;jenkins-hbase4:38597] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK], DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK], DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK]] 2023-07-24 20:11:13,006 DEBUG [RS-EventLoopGroup-16-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK] 2023-07-24 20:11:13,006 DEBUG [RS-EventLoopGroup-16-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK] 2023-07-24 20:11:13,006 DEBUG [RS-EventLoopGroup-16-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK] 2023-07-24 20:11:13,007 DEBUG [RS-EventLoopGroup-16-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK] 2023-07-24 20:11:13,007 WARN [ReadOnlyZKClient-127.0.0.1:62881@0x0bb4fccf] client.ZKConnectionRegistry(168): Meta region is in state OPENING 2023-07-24 20:11:13,007 DEBUG [RS-EventLoopGroup-16-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK] 2023-07-24 20:11:13,007 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,37489,1690229472216] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-24 20:11:13,007 DEBUG [RS-EventLoopGroup-16-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK] 2023-07-24 20:11:13,009 INFO [RS-EventLoopGroup-14-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49552, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-24 20:11:13,009 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46693] ipc.CallRunner(144): callId: 2 service: ClientService methodName: Get size: 88 connection: 172.31.14.131:49552 deadline: 1690229533009, exception=org.apache.hadoop.hbase.NotServingRegionException: hbase:meta,,1 is not online on jenkins-hbase4.apache.org,46693,1690229472273 2023-07-24 20:11:13,015 INFO [RS:0;jenkins-hbase4:46693] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,46693,1690229472273/jenkins-hbase4.apache.org%2C46693%2C1690229472273.1690229472977 2023-07-24 20:11:13,015 DEBUG [RS:0;jenkins-hbase4:46693] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK], DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK], DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK]] 2023-07-24 20:11:13,016 INFO [RS:2;jenkins-hbase4:34773] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,34773,1690229472389/jenkins-hbase4.apache.org%2C34773%2C1690229472389.1690229472977 2023-07-24 20:11:13,016 DEBUG [RS:2;jenkins-hbase4:34773] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK], DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK], DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK]] 2023-07-24 20:11:13,083 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,46693,1690229472273 2023-07-24 20:11:13,085 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-24 20:11:13,086 INFO [RS-EventLoopGroup-14-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49564, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-24 20:11:13,090 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-24 20:11:13,090 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-24 20:11:13,092 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C46693%2C1690229472273.meta, suffix=.meta, logDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,46693,1690229472273, archiveDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/oldWALs, maxLogs=32 2023-07-24 20:11:13,105 DEBUG [RS-EventLoopGroup-16-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK] 2023-07-24 20:11:13,105 DEBUG [RS-EventLoopGroup-16-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK] 2023-07-24 20:11:13,105 DEBUG [RS-EventLoopGroup-16-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK] 2023-07-24 20:11:13,110 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,46693,1690229472273/jenkins-hbase4.apache.org%2C46693%2C1690229472273.meta.1690229473092.meta 2023-07-24 20:11:13,111 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK], DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK], DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK]] 2023-07-24 20:11:13,111 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-24 20:11:13,111 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-24 20:11:13,111 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-24 20:11:13,111 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-24 20:11:13,111 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-24 20:11:13,111 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:11:13,112 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-24 20:11:13,112 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-24 20:11:13,113 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-24 20:11:13,114 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/info 2023-07-24 20:11:13,114 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/info 2023-07-24 20:11:13,114 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-24 20:11:13,121 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/info/14fae0676b7b4b9dbef6a9c00b67e86e 2023-07-24 20:11:13,125 INFO [StoreFileOpener-info-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 71cc04e9686e4212a8a52f78335cc5c6 2023-07-24 20:11:13,126 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/info/71cc04e9686e4212a8a52f78335cc5c6 2023-07-24 20:11:13,130 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/info/99bce8ea8bc2478680bf62bc041d6208 2023-07-24 20:11:13,130 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:11:13,131 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-24 20:11:13,131 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/rep_barrier 2023-07-24 20:11:13,132 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/rep_barrier 2023-07-24 20:11:13,132 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-24 20:11:13,138 INFO [StoreFileOpener-rep_barrier-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d4449076caff44088d8e3a03756fcbd6 2023-07-24 20:11:13,139 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/rep_barrier/d4449076caff44088d8e3a03756fcbd6 2023-07-24 20:11:13,139 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:11:13,139 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-24 20:11:13,140 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/table 2023-07-24 20:11:13,140 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/table 2023-07-24 20:11:13,140 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-24 20:11:13,145 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/table/228b116fcd95447abde6f2baeea5c266 2023-07-24 20:11:13,150 INFO [StoreFileOpener-table-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 92a06725a50449fa925e17781e43f1e3 2023-07-24 20:11:13,150 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/table/92a06725a50449fa925e17781e43f1e3 2023-07-24 20:11:13,157 DEBUG [StoreOpener-1588230740-1] regionserver.HStore(539): loaded hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/table/a90c3b62969d4417a8b965913cf8fbbc 2023-07-24 20:11:13,157 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:11:13,158 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740 2023-07-24 20:11:13,159 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740 2023-07-24 20:11:13,162 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-24 20:11:13,163 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-24 20:11:13,164 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=159; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=10993596640, jitterRate=0.02385847270488739}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-24 20:11:13,164 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-24 20:11:13,165 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=126, masterSystemTime=1690229473083 2023-07-24 20:11:13,167 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 20:11:13,168 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-24 20:11:13,170 DEBUG [RS:0;jenkins-hbase4:46693-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-07-24 20:11:13,170 DEBUG [RS:0;jenkins-hbase4:46693-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-07-24 20:11:13,173 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-24 20:11:13,173 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-24 20:11:13,174 INFO [PEWorker-4] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,46693,1690229472273, state=OPEN 2023-07-24 20:11:13,175 DEBUG [RS:0;jenkins-hbase4:46693-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 16766 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-07-24 20:11:13,175 DEBUG [RS:0;jenkins-hbase4:46693-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 23455 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-07-24 20:11:13,175 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-07-24 20:11:13,176 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-24 20:11:13,177 DEBUG [RS:0;jenkins-hbase4:46693-longCompactions-0] regionserver.HStore(1912): 1588230740/table is initiating minor compaction (all files) 2023-07-24 20:11:13,177 DEBUG [RS:0;jenkins-hbase4:46693-shortCompactions-0] regionserver.HStore(1912): 1588230740/info is initiating minor compaction (all files) 2023-07-24 20:11:13,178 INFO [RS:0;jenkins-hbase4:46693-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 1588230740/info in hbase:meta,,1.1588230740 2023-07-24 20:11:13,178 INFO [RS:0;jenkins-hbase4:46693-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 1588230740/table in hbase:meta,,1.1588230740 2023-07-24 20:11:13,178 INFO [RS:0;jenkins-hbase4:46693-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/info/14fae0676b7b4b9dbef6a9c00b67e86e, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/info/71cc04e9686e4212a8a52f78335cc5c6, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/info/99bce8ea8bc2478680bf62bc041d6208] into tmpdir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/.tmp, totalSize=22.9 K 2023-07-24 20:11:13,178 INFO [RS:0;jenkins-hbase4:46693-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/table/a90c3b62969d4417a8b965913cf8fbbc, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/table/92a06725a50449fa925e17781e43f1e3, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/table/228b116fcd95447abde6f2baeea5c266] into tmpdir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/.tmp, totalSize=16.4 K 2023-07-24 20:11:13,179 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-07-24 20:11:13,179 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,46693,1690229472273 in 256 msec 2023-07-24 20:11:13,179 DEBUG [RS:0;jenkins-hbase4:46693-longCompactions-0] compactions.Compactor(207): Compacting a90c3b62969d4417a8b965913cf8fbbc, keycount=4, bloomtype=NONE, size=4.8 K, encoding=NONE, compression=NONE, seqNum=14, earliestPutTs=1690229442687 2023-07-24 20:11:13,179 DEBUG [RS:0;jenkins-hbase4:46693-shortCompactions-0] compactions.Compactor(207): Compacting 14fae0676b7b4b9dbef6a9c00b67e86e, keycount=20, bloomtype=NONE, size=7.0 K, encoding=NONE, compression=NONE, seqNum=14, earliestPutTs=1690229442628 2023-07-24 20:11:13,180 DEBUG [RS:0;jenkins-hbase4:46693-longCompactions-0] compactions.Compactor(207): Compacting 92a06725a50449fa925e17781e43f1e3, keycount=23, bloomtype=NONE, size=7.0 K, encoding=NONE, compression=NONE, seqNum=140, earliestPutTs=9223372036854775807 2023-07-24 20:11:13,180 DEBUG [RS:0;jenkins-hbase4:46693-shortCompactions-0] compactions.Compactor(207): Compacting 71cc04e9686e4212a8a52f78335cc5c6, keycount=32, bloomtype=NONE, size=8.2 K, encoding=NONE, compression=NONE, seqNum=140, earliestPutTs=9223372036854775807 2023-07-24 20:11:13,180 DEBUG [RS:0;jenkins-hbase4:46693-longCompactions-0] compactions.Compactor(207): Compacting 228b116fcd95447abde6f2baeea5c266, keycount=2, bloomtype=NONE, size=4.7 K, encoding=NONE, compression=NONE, seqNum=155, earliestPutTs=1690229466070 2023-07-24 20:11:13,180 DEBUG [RS:0;jenkins-hbase4:46693-shortCompactions-0] compactions.Compactor(207): Compacting 99bce8ea8bc2478680bf62bc041d6208, keycount=26, bloomtype=NONE, size=7.7 K, encoding=NONE, compression=NONE, seqNum=155, earliestPutTs=1690229464973 2023-07-24 20:11:13,181 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=125, resume processing ppid=124 2023-07-24 20:11:13,181 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, ppid=124, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 422 msec 2023-07-24 20:11:13,207 INFO [RS:0;jenkins-hbase4:46693-longCompactions-0] throttle.PressureAwareThroughputController(145): 1588230740#table#compaction#12 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-07-24 20:11:13,207 INFO [RS:0;jenkins-hbase4:46693-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1588230740#info#compaction#13 average throughput is 4.26 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 20:11:13,234 DEBUG [RS:0;jenkins-hbase4:46693-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/.tmp/table/a018a6792d0f410bb8c019b4518ca4bf as hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/table/a018a6792d0f410bb8c019b4518ca4bf 2023-07-24 20:11:13,239 DEBUG [RS:0;jenkins-hbase4:46693-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/.tmp/info/6547f79ddcd349919e99c61e7df2ac10 as hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/info/6547f79ddcd349919e99c61e7df2ac10 2023-07-24 20:11:13,250 DEBUG [RS:0;jenkins-hbase4:46693-shortCompactions-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-24 20:11:13,250 DEBUG [RS:0;jenkins-hbase4:46693-longCompactions-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-24 20:11:13,253 INFO [RS:0;jenkins-hbase4:46693-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 1588230740/info of 1588230740 into 6547f79ddcd349919e99c61e7df2ac10(size=9.2 K), total size for store is 9.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 20:11:13,253 INFO [RS:0;jenkins-hbase4:46693-longCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 1588230740/table of 1588230740 into a018a6792d0f410bb8c019b4518ca4bf(size=4.9 K), total size for store is 4.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 20:11:13,253 DEBUG [RS:0;jenkins-hbase4:46693-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1588230740: 2023-07-24 20:11:13,253 DEBUG [RS:0;jenkins-hbase4:46693-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1588230740: 2023-07-24 20:11:13,253 INFO [RS:0;jenkins-hbase4:46693-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=hbase:meta,,1.1588230740, storeName=1588230740/info, priority=13, startTime=1690229473167; duration=0sec 2023-07-24 20:11:13,253 INFO [RS:0;jenkins-hbase4:46693-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=hbase:meta,,1.1588230740, storeName=1588230740/table, priority=13, startTime=1690229473168; duration=0sec 2023-07-24 20:11:13,254 DEBUG [RS:0;jenkins-hbase4:46693-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 20:11:13,254 DEBUG [RS:0;jenkins-hbase4:46693-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 20:11:13,322 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,37489,1690229472216] ipc.AbstractRpcClient(347): Not trying to connect to jenkins-hbase4.apache.org/172.31.14.131:46705 this server is in the failed servers list 2023-07-24 20:11:13,428 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,37489,1690229472216] ipc.AbstractRpcClient(347): Not trying to connect to jenkins-hbase4.apache.org/172.31.14.131:46705 this server is in the failed servers list 2023-07-24 20:11:13,634 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,37489,1690229472216] ipc.AbstractRpcClient(347): Not trying to connect to jenkins-hbase4.apache.org/172.31.14.131:46705 this server is in the failed servers list 2023-07-24 20:11:13,945 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,37489,1690229472216] ipc.AbstractRpcClient(347): Not trying to connect to jenkins-hbase4.apache.org/172.31.14.131:46705 this server is in the failed servers list 2023-07-24 20:11:14,294 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=3; waited=1553ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=1503ms 2023-07-24 20:11:14,450 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,37489,1690229472216] ipc.AbstractRpcClient(347): Not trying to connect to jenkins-hbase4.apache.org/172.31.14.131:46705 this server is in the failed servers list 2023-07-24 20:11:15,463 WARN [RS-EventLoopGroup-16-3] ipc.NettyRpcConnection$2(294): Exception encountered while connecting to the server jenkins-hbase4.apache.org/172.31.14.131:46705 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:46705 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:155) at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:128) at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-07-24 20:11:15,464 DEBUG [RS-EventLoopGroup-16-3] ipc.FailedServers(52): Added failed server with address jenkins-hbase4.apache.org/172.31.14.131:46705 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:46705 2023-07-24 20:11:15,797 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(801): Waiting on regionserver count=3; waited=3056ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=3006ms 2023-07-24 20:11:17,257 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=4516ms, expected min=1 server(s), max=NO_LIMIT server(s), master is running 2023-07-24 20:11:17,258 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 20:11:17,261 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=324410cf2a29c2dca1a19e8570799d47, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46065,1690229460063, regionLocation=jenkins-hbase4.apache.org,46065,1690229460063, openSeqNum=15 2023-07-24 20:11:17,261 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=4fa2a21e5a49480a149a055b0fc39515, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46065,1690229460063, regionLocation=jenkins-hbase4.apache.org,46065,1690229460063, openSeqNum=2 2023-07-24 20:11:17,261 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.RegionStateStore(147): Load hbase:meta entry region=ec4f892f95beff2c1aa7863da43a18fa, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,46705,1690229459880, regionLocation=jenkins-hbase4.apache.org,46705,1690229459880, openSeqNum=71 2023-07-24 20:11:17,261 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 20:11:17,261 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690229537261 2023-07-24 20:11:17,261 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690229597261 2023-07-24 20:11:17,261 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-24 20:11:17,281 INFO [PEWorker-5] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,46705,1690229459880 had 2 regions 2023-07-24 20:11:17,281 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37489,1690229472216-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:17,281 INFO [PEWorker-2] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,46065,1690229460063 had 2 regions 2023-07-24 20:11:17,281 INFO [PEWorker-4] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,45019,1690229459931 had 0 regions 2023-07-24 20:11:17,281 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37489,1690229472216-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:17,281 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37489,1690229472216-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:17,281 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:37489, period=300000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:17,281 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:17,282 WARN [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1240): hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. is NOT online; state={324410cf2a29c2dca1a19e8570799d47 state=OPEN, ts=1690229477261, server=jenkins-hbase4.apache.org,46065,1690229460063}; ServerCrashProcedures=true. Master startup cannot progress, in holding-pattern until region onlined. 2023-07-24 20:11:17,282 INFO [PEWorker-2] procedure.ServerCrashProcedure(300): Splitting WALs pid=123, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,46065,1690229460063, splitWal=true, meta=false, isMeta: false 2023-07-24 20:11:17,283 INFO [PEWorker-5] procedure.ServerCrashProcedure(300): Splitting WALs pid=124, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,46705,1690229459880, splitWal=true, meta=true, isMeta: false 2023-07-24 20:11:17,282 INFO [PEWorker-4] procedure.ServerCrashProcedure(300): Splitting WALs pid=122, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,45019,1690229459931, splitWal=true, meta=false, isMeta: false 2023-07-24 20:11:17,284 DEBUG [PEWorker-2] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,46065,1690229460063-splitting 2023-07-24 20:11:17,286 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,46065,1690229460063-splitting dir is empty, no logs to split. 2023-07-24 20:11:17,286 WARN [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(172): unknown_server=jenkins-hbase4.apache.org,46065,1690229460063/hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47., unknown_server=jenkins-hbase4.apache.org,46065,1690229460063/hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515., unknown_server=jenkins-hbase4.apache.org,46705,1690229459880/hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. 2023-07-24 20:11:17,286 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase4.apache.org,46065,1690229460063 WAL count=0, meta=false 2023-07-24 20:11:17,287 DEBUG [PEWorker-4] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,45019,1690229459931-splitting 2023-07-24 20:11:17,288 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,45019,1690229459931-splitting dir is empty, no logs to split. 2023-07-24 20:11:17,288 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,45019,1690229459931 WAL count=0, meta=false 2023-07-24 20:11:17,289 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,46705,1690229459880-splitting dir is empty, no logs to split. 2023-07-24 20:11:17,289 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,46705,1690229459880 WAL count=0, meta=false 2023-07-24 20:11:17,291 INFO [PEWorker-2] master.SplitLogManager(171): hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,46065,1690229460063-splitting dir is empty, no logs to split. 2023-07-24 20:11:17,291 INFO [PEWorker-2] master.SplitWALManager(106): jenkins-hbase4.apache.org,46065,1690229460063 WAL count=0, meta=false 2023-07-24 20:11:17,292 DEBUG [PEWorker-2] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,46065,1690229460063 WAL splitting is done? wals=0, meta=false 2023-07-24 20:11:17,292 INFO [PEWorker-4] master.SplitLogManager(171): hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,45019,1690229459931-splitting dir is empty, no logs to split. 2023-07-24 20:11:17,293 INFO [PEWorker-4] master.SplitWALManager(106): jenkins-hbase4.apache.org,45019,1690229459931 WAL count=0, meta=false 2023-07-24 20:11:17,293 DEBUG [PEWorker-4] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,45019,1690229459931 WAL splitting is done? wals=0, meta=false 2023-07-24 20:11:17,295 INFO [PEWorker-5] master.SplitLogManager(171): hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,46705,1690229459880-splitting dir is empty, no logs to split. 2023-07-24 20:11:17,295 INFO [PEWorker-5] master.SplitWALManager(106): jenkins-hbase4.apache.org,46705,1690229459880 WAL count=0, meta=false 2023-07-24 20:11:17,295 DEBUG [PEWorker-5] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,46705,1690229459880 WAL splitting is done? wals=0, meta=false 2023-07-24 20:11:17,295 INFO [PEWorker-2] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,46065,1690229460063 failed, ignore...File hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,46065,1690229460063-splitting does not exist. 2023-07-24 20:11:17,297 INFO [PEWorker-4] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,45019,1690229459931 failed, ignore...File hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,45019,1690229459931-splitting does not exist. 2023-07-24 20:11:17,297 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=124, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:rsgroup, region=ec4f892f95beff2c1aa7863da43a18fa, ASSIGN}] 2023-07-24 20:11:17,297 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=128, ppid=123, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:quota, region=4fa2a21e5a49480a149a055b0fc39515, ASSIGN}, {pid=129, ppid=123, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=324410cf2a29c2dca1a19e8570799d47, ASSIGN}] 2023-07-24 20:11:17,298 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=127, ppid=124, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:rsgroup, region=ec4f892f95beff2c1aa7863da43a18fa, ASSIGN 2023-07-24 20:11:17,298 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, ppid=123, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:quota, region=4fa2a21e5a49480a149a055b0fc39515, ASSIGN 2023-07-24 20:11:17,298 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=123, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=324410cf2a29c2dca1a19e8570799d47, ASSIGN 2023-07-24 20:11:17,299 INFO [PEWorker-4] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,45019,1690229459931 after splitting done 2023-07-24 20:11:17,299 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=127, ppid=124, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:rsgroup, region=ec4f892f95beff2c1aa7863da43a18fa, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-24 20:11:17,299 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=128, ppid=123, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:quota, region=4fa2a21e5a49480a149a055b0fc39515, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-24 20:11:17,299 DEBUG [PEWorker-4] master.DeadServer(114): Removed jenkins-hbase4.apache.org,45019,1690229459931 from processing; numProcessing=2 2023-07-24 20:11:17,299 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=129, ppid=123, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=324410cf2a29c2dca1a19e8570799d47, ASSIGN; state=OPEN, location=null; forceNewPlan=true, retain=false 2023-07-24 20:11:17,299 DEBUG [jenkins-hbase4:37489] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-07-24 20:11:17,300 DEBUG [jenkins-hbase4:37489] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-07-24 20:11:17,300 DEBUG [jenkins-hbase4:37489] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 20:11:17,300 DEBUG [jenkins-hbase4:37489] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 20:11:17,300 DEBUG [jenkins-hbase4:37489] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 20:11:17,300 DEBUG [jenkins-hbase4:37489] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-07-24 20:11:17,303 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=324410cf2a29c2dca1a19e8570799d47, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,38597,1690229472335 2023-07-24 20:11:17,303 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=128 updating hbase:meta row=4fa2a21e5a49480a149a055b0fc39515, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,34773,1690229472389 2023-07-24 20:11:17,303 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515.","families":{"info":[{"qualifier":"regioninfo","vlen":37,"tag":[],"timestamp":"1690229477303"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229477303"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229477303"}]},"ts":"1690229477303"} 2023-07-24 20:11:17,303 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690229477303"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229477303"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229477303"}]},"ts":"1690229477303"} 2023-07-24 20:11:17,303 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=127 updating hbase:meta row=ec4f892f95beff2c1aa7863da43a18fa, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46693,1690229472273 2023-07-24 20:11:17,304 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa.","families":{"info":[{"qualifier":"regioninfo","vlen":39,"tag":[],"timestamp":"1690229477303"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229477303"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229477303"}]},"ts":"1690229477303"} 2023-07-24 20:11:17,307 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=122, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,45019,1690229459931, splitWal=true, meta=false in 4.6420 sec 2023-07-24 20:11:17,309 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=128, state=RUNNABLE; OpenRegionProcedure 4fa2a21e5a49480a149a055b0fc39515, server=jenkins-hbase4.apache.org,34773,1690229472389}] 2023-07-24 20:11:17,310 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=129, state=RUNNABLE; OpenRegionProcedure 324410cf2a29c2dca1a19e8570799d47, server=jenkins-hbase4.apache.org,38597,1690229472335}] 2023-07-24 20:11:17,312 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=132, ppid=127, state=RUNNABLE; OpenRegionProcedure ec4f892f95beff2c1aa7863da43a18fa, server=jenkins-hbase4.apache.org,46693,1690229472273}] 2023-07-24 20:11:17,464 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,34773,1690229472389 2023-07-24 20:11:17,464 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,38597,1690229472335 2023-07-24 20:11:17,464 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-24 20:11:17,464 DEBUG [RSProcedureDispatcher-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-24 20:11:17,465 INFO [RS-EventLoopGroup-16-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53150, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-24 20:11:17,465 INFO [RS-EventLoopGroup-15-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43840, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-24 20:11:17,469 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. 2023-07-24 20:11:17,469 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ec4f892f95beff2c1aa7863da43a18fa, NAME => 'hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa.', STARTKEY => '', ENDKEY => ''} 2023-07-24 20:11:17,469 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-24 20:11:17,469 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. service=MultiRowMutationService 2023-07-24 20:11:17,469 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:rsgroup successfully. 2023-07-24 20:11:17,469 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table rsgroup ec4f892f95beff2c1aa7863da43a18fa 2023-07-24 20:11:17,470 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:11:17,470 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ec4f892f95beff2c1aa7863da43a18fa 2023-07-24 20:11:17,470 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ec4f892f95beff2c1aa7863da43a18fa 2023-07-24 20:11:17,470 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. 2023-07-24 20:11:17,470 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 324410cf2a29c2dca1a19e8570799d47, NAME => 'hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47.', STARTKEY => '', ENDKEY => ''} 2023-07-24 20:11:17,470 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 324410cf2a29c2dca1a19e8570799d47 2023-07-24 20:11:17,471 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:11:17,471 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 324410cf2a29c2dca1a19e8570799d47 2023-07-24 20:11:17,471 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 324410cf2a29c2dca1a19e8570799d47 2023-07-24 20:11:17,471 INFO [StoreOpener-ec4f892f95beff2c1aa7863da43a18fa-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family m of region ec4f892f95beff2c1aa7863da43a18fa 2023-07-24 20:11:17,472 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515. 2023-07-24 20:11:17,472 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 4fa2a21e5a49480a149a055b0fc39515, NAME => 'hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515.', STARTKEY => '', ENDKEY => ''} 2023-07-24 20:11:17,472 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table quota 4fa2a21e5a49480a149a055b0fc39515 2023-07-24 20:11:17,472 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:11:17,472 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 4fa2a21e5a49480a149a055b0fc39515 2023-07-24 20:11:17,472 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 4fa2a21e5a49480a149a055b0fc39515 2023-07-24 20:11:17,473 DEBUG [StoreOpener-ec4f892f95beff2c1aa7863da43a18fa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/rsgroup/ec4f892f95beff2c1aa7863da43a18fa/m 2023-07-24 20:11:17,473 DEBUG [StoreOpener-ec4f892f95beff2c1aa7863da43a18fa-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/rsgroup/ec4f892f95beff2c1aa7863da43a18fa/m 2023-07-24 20:11:17,473 INFO [StoreOpener-324410cf2a29c2dca1a19e8570799d47-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 324410cf2a29c2dca1a19e8570799d47 2023-07-24 20:11:17,473 INFO [StoreOpener-ec4f892f95beff2c1aa7863da43a18fa-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ec4f892f95beff2c1aa7863da43a18fa columnFamilyName m 2023-07-24 20:11:17,475 INFO [StoreOpener-4fa2a21e5a49480a149a055b0fc39515-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family q of region 4fa2a21e5a49480a149a055b0fc39515 2023-07-24 20:11:17,475 DEBUG [StoreOpener-324410cf2a29c2dca1a19e8570799d47-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/namespace/324410cf2a29c2dca1a19e8570799d47/info 2023-07-24 20:11:17,475 DEBUG [StoreOpener-324410cf2a29c2dca1a19e8570799d47-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/namespace/324410cf2a29c2dca1a19e8570799d47/info 2023-07-24 20:11:17,475 INFO [StoreOpener-324410cf2a29c2dca1a19e8570799d47-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 324410cf2a29c2dca1a19e8570799d47 columnFamilyName info 2023-07-24 20:11:17,475 WARN [RS-EventLoopGroup-16-3] ipc.NettyRpcConnection$2(294): Exception encountered while connecting to the server jenkins-hbase4.apache.org/172.31.14.131:46705 org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:46705 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:155) at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:128) at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-07-24 20:11:17,477 DEBUG [StoreOpener-4fa2a21e5a49480a149a055b0fc39515-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/quota/4fa2a21e5a49480a149a055b0fc39515/q 2023-07-24 20:11:17,477 DEBUG [StoreOpener-4fa2a21e5a49480a149a055b0fc39515-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/quota/4fa2a21e5a49480a149a055b0fc39515/q 2023-07-24 20:11:17,477 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,37489,1690229472216] client.RpcRetryingCallerImpl(129): Call exception, tries=6, retries=46, started=4160 ms ago, cancelled=false, msg=Call to address=jenkins-hbase4.apache.org/172.31.14.131:46705 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:46705, details=row '\x00' on table 'hbase:rsgroup' at region=hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa., hostname=jenkins-hbase4.apache.org,46705,1690229459880, seqNum=71, see https://s.apache.org/timeout, exception=java.net.ConnectException: Call to address=jenkins-hbase4.apache.org/172.31.14.131:46705 failed on connection exception: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:46705 at org.apache.hadoop.hbase.ipc.IPCUtil.wrapException(IPCUtil.java:186) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:385) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.BufferCallBeforeInitHandler.userEventTriggered(BufferCallBeforeInitHandler.java:99) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:398) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:368) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.userEventTriggered(DefaultChannelPipeline.java:1428) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:396) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:376) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireUserEventTriggered(DefaultChannelPipeline.java:913) at org.apache.hadoop.hbase.ipc.NettyRpcConnection.failInit(NettyRpcConnection.java:195) at org.apache.hadoop.hbase.ipc.NettyRpcConnection.access$300(NettyRpcConnection.java:76) at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:296) at org.apache.hadoop.hbase.ipc.NettyRpcConnection$2.operationComplete(NettyRpcConnection.java:287) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:629) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:118) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:674) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:693) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:46705 Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.newConnectException0(Errors.java:155) at org.apache.hbase.thirdparty.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:128) at org.apache.hbase.thirdparty.io.netty.channel.unix.Socket.finishConnect(Socket.java:359) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-07-24 20:11:17,477 DEBUG [RS-EventLoopGroup-16-3] ipc.FailedServers(52): Added failed server with address jenkins-hbase4.apache.org/172.31.14.131:46705 to list caused by org.apache.hbase.thirdparty.io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: jenkins-hbase4.apache.org/172.31.14.131:46705 2023-07-24 20:11:17,477 INFO [StoreOpener-4fa2a21e5a49480a149a055b0fc39515-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4fa2a21e5a49480a149a055b0fc39515 columnFamilyName q 2023-07-24 20:11:17,478 INFO [StoreOpener-4fa2a21e5a49480a149a055b0fc39515-1] regionserver.HStore(310): Store=4fa2a21e5a49480a149a055b0fc39515/q, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:11:17,478 INFO [StoreOpener-4fa2a21e5a49480a149a055b0fc39515-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family u of region 4fa2a21e5a49480a149a055b0fc39515 2023-07-24 20:11:17,478 DEBUG [StoreOpener-4fa2a21e5a49480a149a055b0fc39515-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/quota/4fa2a21e5a49480a149a055b0fc39515/u 2023-07-24 20:11:17,479 DEBUG [StoreOpener-4fa2a21e5a49480a149a055b0fc39515-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/quota/4fa2a21e5a49480a149a055b0fc39515/u 2023-07-24 20:11:17,479 INFO [StoreOpener-4fa2a21e5a49480a149a055b0fc39515-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4fa2a21e5a49480a149a055b0fc39515 columnFamilyName u 2023-07-24 20:11:17,479 INFO [StoreOpener-4fa2a21e5a49480a149a055b0fc39515-1] regionserver.HStore(310): Store=4fa2a21e5a49480a149a055b0fc39515/u, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:11:17,480 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/quota/4fa2a21e5a49480a149a055b0fc39515 2023-07-24 20:11:17,481 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/quota/4fa2a21e5a49480a149a055b0fc39515 2023-07-24 20:11:17,486 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:quota descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 20:11:17,486 INFO [StoreFileOpener-m-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 327d57459138467dbba127685af878e9 2023-07-24 20:11:17,486 INFO [StoreFileOpener-info-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 81b8365c87814c2e9ac1bfab2b6bb254 2023-07-24 20:11:17,487 DEBUG [StoreOpener-ec4f892f95beff2c1aa7863da43a18fa-1] regionserver.HStore(539): loaded hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/rsgroup/ec4f892f95beff2c1aa7863da43a18fa/m/327d57459138467dbba127685af878e9 2023-07-24 20:11:17,487 DEBUG [StoreOpener-324410cf2a29c2dca1a19e8570799d47-1] regionserver.HStore(539): loaded hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/namespace/324410cf2a29c2dca1a19e8570799d47/info/81b8365c87814c2e9ac1bfab2b6bb254 2023-07-24 20:11:17,487 INFO [StoreOpener-324410cf2a29c2dca1a19e8570799d47-1] regionserver.HStore(310): Store=324410cf2a29c2dca1a19e8570799d47/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:11:17,488 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/namespace/324410cf2a29c2dca1a19e8570799d47 2023-07-24 20:11:17,489 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/namespace/324410cf2a29c2dca1a19e8570799d47 2023-07-24 20:11:17,493 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 324410cf2a29c2dca1a19e8570799d47 2023-07-24 20:11:17,493 DEBUG [StoreOpener-ec4f892f95beff2c1aa7863da43a18fa-1] regionserver.HStore(539): loaded hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/rsgroup/ec4f892f95beff2c1aa7863da43a18fa/m/cfb13056081245e68f9511bc8b4d0363 2023-07-24 20:11:17,493 INFO [StoreOpener-ec4f892f95beff2c1aa7863da43a18fa-1] regionserver.HStore(310): Store=ec4f892f95beff2c1aa7863da43a18fa/m, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:11:17,494 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/rsgroup/ec4f892f95beff2c1aa7863da43a18fa 2023-07-24 20:11:17,494 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 324410cf2a29c2dca1a19e8570799d47; next sequenceid=18; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=11596717920, jitterRate=0.08002851903438568}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 20:11:17,494 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 324410cf2a29c2dca1a19e8570799d47: 2023-07-24 20:11:17,495 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47., pid=131, masterSystemTime=1690229477464 2023-07-24 20:11:17,498 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 4fa2a21e5a49480a149a055b0fc39515 2023-07-24 20:11:17,499 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/rsgroup/ec4f892f95beff2c1aa7863da43a18fa 2023-07-24 20:11:17,501 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. 2023-07-24 20:11:17,501 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=324410cf2a29c2dca1a19e8570799d47, regionState=OPEN, openSeqNum=18, regionLocation=jenkins-hbase4.apache.org,38597,1690229472335 2023-07-24 20:11:17,502 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. 2023-07-24 20:11:17,501 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 4fa2a21e5a49480a149a055b0fc39515; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=11876564320, jitterRate=0.10609124600887299}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 20:11:17,504 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690229477501"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690229477501"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690229477501"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690229477501"}]},"ts":"1690229477501"} 2023-07-24 20:11:17,504 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 4fa2a21e5a49480a149a055b0fc39515: 2023-07-24 20:11:17,505 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ec4f892f95beff2c1aa7863da43a18fa 2023-07-24 20:11:17,505 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515., pid=130, masterSystemTime=1690229477464 2023-07-24 20:11:17,508 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ec4f892f95beff2c1aa7863da43a18fa; next sequenceid=78; org.apache.hadoop.hbase.regionserver.DisabledRegionSplitPolicy@4bfe5bc4, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 20:11:17,508 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ec4f892f95beff2c1aa7863da43a18fa: 2023-07-24 20:11:17,509 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa., pid=132, masterSystemTime=1690229477464 2023-07-24 20:11:17,510 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515. 2023-07-24 20:11:17,511 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515. 2023-07-24 20:11:17,511 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=131, resume processing ppid=129 2023-07-24 20:11:17,511 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=129, state=SUCCESS; OpenRegionProcedure 324410cf2a29c2dca1a19e8570799d47, server=jenkins-hbase4.apache.org,38597,1690229472335 in 197 msec 2023-07-24 20:11:17,511 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=128 updating hbase:meta row=4fa2a21e5a49480a149a055b0fc39515, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,34773,1690229472389 2023-07-24 20:11:17,511 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515.","families":{"info":[{"qualifier":"regioninfo","vlen":37,"tag":[],"timestamp":"1690229477511"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690229477511"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690229477511"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690229477511"}]},"ts":"1690229477511"} 2023-07-24 20:11:17,511 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. 2023-07-24 20:11:17,511 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. 2023-07-24 20:11:17,512 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=127 updating hbase:meta row=ec4f892f95beff2c1aa7863da43a18fa, regionState=OPEN, openSeqNum=78, regionLocation=jenkins-hbase4.apache.org,46693,1690229472273 2023-07-24 20:11:17,512 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa.","families":{"info":[{"qualifier":"regioninfo","vlen":39,"tag":[],"timestamp":"1690229477512"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690229477512"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690229477512"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690229477512"}]},"ts":"1690229477512"} 2023-07-24 20:11:17,513 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=123, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=324410cf2a29c2dca1a19e8570799d47, ASSIGN in 214 msec 2023-07-24 20:11:17,515 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=128 2023-07-24 20:11:17,515 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=128, state=SUCCESS; OpenRegionProcedure 4fa2a21e5a49480a149a055b0fc39515, server=jenkins-hbase4.apache.org,34773,1690229472389 in 204 msec 2023-07-24 20:11:17,515 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=132, resume processing ppid=127 2023-07-24 20:11:17,515 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=127, state=SUCCESS; OpenRegionProcedure ec4f892f95beff2c1aa7863da43a18fa, server=jenkins-hbase4.apache.org,46693,1690229472273 in 202 msec 2023-07-24 20:11:17,516 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=128, resume processing ppid=123 2023-07-24 20:11:17,516 INFO [PEWorker-5] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,46065,1690229460063 after splitting done 2023-07-24 20:11:17,516 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=128, ppid=123, state=SUCCESS; TransitRegionStateProcedure table=hbase:quota, region=4fa2a21e5a49480a149a055b0fc39515, ASSIGN in 218 msec 2023-07-24 20:11:17,516 DEBUG [PEWorker-5] master.DeadServer(114): Removed jenkins-hbase4.apache.org,46065,1690229460063 from processing; numProcessing=1 2023-07-24 20:11:17,517 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=124 2023-07-24 20:11:17,517 INFO [PEWorker-3] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,46705,1690229459880 after splitting done 2023-07-24 20:11:17,517 DEBUG [PEWorker-3] master.DeadServer(114): Removed jenkins-hbase4.apache.org,46705,1690229459880 from processing; numProcessing=0 2023-07-24 20:11:17,517 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=124, state=SUCCESS; TransitRegionStateProcedure table=hbase:rsgroup, region=ec4f892f95beff2c1aa7863da43a18fa, ASSIGN in 218 msec 2023-07-24 20:11:17,518 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=123, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,46065,1690229460063, splitWal=true, meta=false in 4.8570 sec 2023-07-24 20:11:17,519 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,46705,1690229459880, splitWal=true, meta=true in 4.8530 sec 2023-07-24 20:11:17,892 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: Master,sub=Coprocessor.Master.CP_org.apache.hadoop.hbase.quotas.MasterQuotasObserver 2023-07-24 20:11:18,282 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/namespace 2023-07-24 20:11:18,288 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-24 20:11:18,290 INFO [RS-EventLoopGroup-15-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43856, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-24 20:11:18,307 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-07-24 20:11:18,309 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-07-24 20:11:18,309 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 5.853sec 2023-07-24 20:11:18,309 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-24 20:11:18,309 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-24 20:11:18,309 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-24 20:11:18,309 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37489,1690229472216-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-24 20:11:18,310 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37489,1690229472216-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-24 20:11:18,310 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-24 20:11:18,354 DEBUG [Listener at localhost/44145] zookeeper.ReadOnlyZKClient(139): Connect 0x75844726 to 127.0.0.1:62881 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-24 20:11:18,360 DEBUG [Listener at localhost/44145] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@770f3601, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-24 20:11:18,362 DEBUG [hconnection-0xcd10542-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-24 20:11:18,364 INFO [RS-EventLoopGroup-14-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:49580, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-24 20:11:18,369 INFO [Listener at localhost/44145] hbase.HBaseTestingUtility(1262): HBase has been restarted 2023-07-24 20:11:18,370 DEBUG [Listener at localhost/44145] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x75844726 to 127.0.0.1:62881 2023-07-24 20:11:18,370 DEBUG [Listener at localhost/44145] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:11:18,371 INFO [Listener at localhost/44145] hbase.HBaseTestingUtility(2939): Invalidated connection. Updating master addresses before: jenkins-hbase4.apache.org:37489 after: jenkins-hbase4.apache.org:37489 2023-07-24 20:11:18,372 DEBUG [Listener at localhost/44145] zookeeper.ReadOnlyZKClient(139): Connect 0x5e38523b to 127.0.0.1:62881 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-24 20:11:18,377 DEBUG [Listener at localhost/44145] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5ac7b7ab, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-24 20:11:18,377 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:11:18,616 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-24 20:11:18,805 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:quota' 2023-07-24 20:11:18,806 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-24 20:11:18,807 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:rsgroup' 2023-07-24 20:11:21,493 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,37489,1690229472216] rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker(840): RSGroup table=hbase:rsgroup is online, refreshing cached information 2023-07-24 20:11:21,493 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,37489,1690229472216] rsgroup.RSGroupInfoManagerImpl(534): Refreshing in Online mode. 2023-07-24 20:11:21,503 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,37489,1690229472216] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:11:21,504 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,37489,1690229472216] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:11:21,504 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,37489,1690229472216] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-24 20:11:21,506 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rsgroup 2023-07-24 20:11:21,506 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker-jenkins-hbase4.apache.org,37489,1690229472216] rsgroup.RSGroupInfoManagerImpl$RSGroupStartupWorker(823): GroupBasedLoadBalancer is now online 2023-07-24 20:11:21,581 DEBUG [Listener at localhost/44145] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-07-24 20:11:21,590 INFO [RS-EventLoopGroup-13-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58688, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-07-24 20:11:21,592 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-24 20:11:21,593 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-07-24 20:11:21,593 DEBUG [Listener at localhost/44145] zookeeper.ReadOnlyZKClient(139): Connect 0x45261108 to 127.0.0.1:62881 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-24 20:11:21,610 DEBUG [Listener at localhost/44145] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5b7929c8, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-24 20:11:21,610 INFO [Listener at localhost/44145] zookeeper.RecoverableZooKeeper(93): Process identifier=VerifyingRSGroupAdminClient connecting to ZooKeeper ensemble=127.0.0.1:62881 2023-07-24 20:11:21,618 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): VerifyingRSGroupAdminClient0x0, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-24 20:11:21,621 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:11:21,624 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:11:21,624 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): VerifyingRSGroupAdminClient-0x10198f4f3ce0027 connected 2023-07-24 20:11:21,626 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-24 20:11:21,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-24 20:11:21,626 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-24 20:11:21,627 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-24 20:11:21,627 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-24 20:11:21,628 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-24 20:11:21,632 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:11:21,632 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-24 20:11:21,633 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-24 20:11:21,636 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase(152): Restoring servers: 1 2023-07-24 20:11:21,649 INFO [Listener at localhost/44145] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-24 20:11:21,649 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 20:11:21,650 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-24 20:11:21,650 INFO [Listener at localhost/44145] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-24 20:11:21,650 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 20:11:21,650 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-24 20:11:21,650 INFO [Listener at localhost/44145] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-24 20:11:21,651 INFO [Listener at localhost/44145] ipc.NettyRpcServer(120): Bind to /172.31.14.131:41283 2023-07-24 20:11:21,651 INFO [Listener at localhost/44145] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-24 20:11:21,653 DEBUG [Listener at localhost/44145] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-24 20:11:21,654 INFO [Listener at localhost/44145] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:11:21,655 INFO [Listener at localhost/44145] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:11:21,657 INFO [Listener at localhost/44145] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:41283 connecting to ZooKeeper ensemble=127.0.0.1:62881 2023-07-24 20:11:21,663 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:412830x0, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-24 20:11:21,674 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(162): regionserver:412830x0, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-24 20:11:21,675 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:41283-0x10198f4f3ce0028 connected 2023-07-24 20:11:21,676 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(162): regionserver:41283-0x10198f4f3ce0028, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/running 2023-07-24 20:11:21,677 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): regionserver:41283-0x10198f4f3ce0028, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-24 20:11:21,678 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41283 2023-07-24 20:11:21,678 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41283 2023-07-24 20:11:21,683 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41283 2023-07-24 20:11:21,684 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41283 2023-07-24 20:11:21,684 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41283 2023-07-24 20:11:21,687 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2023-07-24 20:11:21,687 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter) 2023-07-24 20:11:21,687 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'securityheaders' (class=org.apache.hadoop.hbase.http.SecurityHeadersFilter) 2023-07-24 20:11:21,687 INFO [Listener at localhost/44145] http.HttpServer(879): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context regionserver 2023-07-24 20:11:21,688 INFO [Listener at localhost/44145] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2023-07-24 20:11:21,688 INFO [Listener at localhost/44145] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2023-07-24 20:11:21,688 INFO [Listener at localhost/44145] http.HttpServer(783): ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint. 2023-07-24 20:11:21,689 INFO [Listener at localhost/44145] http.HttpServer(1146): Jetty bound to port 41237 2023-07-24 20:11:21,689 INFO [Listener at localhost/44145] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-24 20:11:21,702 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:11:21,702 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@4c9fd641{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/hadoop.log.dir/,AVAILABLE} 2023-07-24 20:11:21,702 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:11:21,703 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@2babb338{static,/static,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/static/,AVAILABLE} 2023-07-24 20:11:21,709 INFO [Listener at localhost/44145] webapp.StandardDescriptorProcessor(277): NO JSP Support for /, did not find org.apache.hbase.thirdparty.org.eclipse.jetty.jsp.JettyJspServlet 2023-07-24 20:11:21,710 INFO [Listener at localhost/44145] session.DefaultSessionIdManager(334): DefaultSessionIdManager workerName=node0 2023-07-24 20:11:21,710 INFO [Listener at localhost/44145] session.DefaultSessionIdManager(339): No SessionScavenger set, using defaults 2023-07-24 20:11:21,711 INFO [Listener at localhost/44145] session.HouseKeeper(132): node0 Scavenging every 660000ms 2023-07-24 20:11:21,711 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:11:21,712 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.w.WebAppContext@66477fc4{regionserver,/,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver/,AVAILABLE}{file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver} 2023-07-24 20:11:21,714 INFO [Listener at localhost/44145] server.AbstractConnector(333): Started ServerConnector@76b99374{HTTP/1.1, (http/1.1)}{0.0.0.0:41237} 2023-07-24 20:11:21,714 INFO [Listener at localhost/44145] server.Server(415): Started @50281ms 2023-07-24 20:11:21,720 INFO [RS:3;jenkins-hbase4:41283] regionserver.HRegionServer(951): ClusterId : dc54b028-3544-4133-8f0e-d550a19bc790 2023-07-24 20:11:21,720 DEBUG [RS:3;jenkins-hbase4:41283] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-24 20:11:21,724 DEBUG [RS:3;jenkins-hbase4:41283] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-24 20:11:21,724 DEBUG [RS:3;jenkins-hbase4:41283] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-24 20:11:21,726 DEBUG [RS:3;jenkins-hbase4:41283] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-24 20:11:21,729 DEBUG [RS:3;jenkins-hbase4:41283] zookeeper.ReadOnlyZKClient(139): Connect 0x0621c7c0 to 127.0.0.1:62881 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-24 20:11:21,734 DEBUG [RS:3;jenkins-hbase4:41283] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@8e62974, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-24 20:11:21,734 DEBUG [RS:3;jenkins-hbase4:41283] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@cd9acb0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-24 20:11:21,746 DEBUG [RS:3;jenkins-hbase4:41283] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:3;jenkins-hbase4:41283 2023-07-24 20:11:21,746 INFO [RS:3;jenkins-hbase4:41283] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-24 20:11:21,746 INFO [RS:3;jenkins-hbase4:41283] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-24 20:11:21,746 DEBUG [RS:3;jenkins-hbase4:41283] regionserver.HRegionServer(1022): About to register with Master. 2023-07-24 20:11:21,746 INFO [RS:3;jenkins-hbase4:41283] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,37489,1690229472216 with isa=jenkins-hbase4.apache.org/172.31.14.131:41283, startcode=1690229481649 2023-07-24 20:11:21,747 DEBUG [RS:3;jenkins-hbase4:41283] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-24 20:11:21,749 INFO [RS-EventLoopGroup-13-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36869, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.11 (auth:SIMPLE), service=RegionServerStatusService 2023-07-24 20:11:21,750 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=37489] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,41283,1690229481649 2023-07-24 20:11:21,750 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,37489,1690229472216] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(787): Updating default servers. 2023-07-24 20:11:21,750 DEBUG [RS:3;jenkins-hbase4:41283] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4 2023-07-24 20:11:21,750 DEBUG [RS:3;jenkins-hbase4:41283] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:37609 2023-07-24 20:11:21,750 DEBUG [RS:3;jenkins-hbase4:41283] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=43499 2023-07-24 20:11:21,752 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:38597-0x10198f4f3ce001e, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:11:21,752 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:46693-0x10198f4f3ce001d, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:11:21,752 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:11:21,752 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34773-0x10198f4f3ce001f, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:11:21,753 DEBUG [RS:3;jenkins-hbase4:41283] zookeeper.ZKUtil(162): regionserver:41283-0x10198f4f3ce0028, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41283,1690229481649 2023-07-24 20:11:21,753 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:46693-0x10198f4f3ce001d, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34773,1690229472389 2023-07-24 20:11:21,753 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:38597-0x10198f4f3ce001e, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34773,1690229472389 2023-07-24 20:11:21,753 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34773-0x10198f4f3ce001f, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34773,1690229472389 2023-07-24 20:11:21,753 WARN [RS:3;jenkins-hbase4:41283] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-24 20:11:21,753 INFO [RS:3;jenkins-hbase4:41283] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-24 20:11:21,753 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,41283,1690229481649] 2023-07-24 20:11:21,753 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,37489,1690229472216] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:11:21,753 DEBUG [RS:3;jenkins-hbase4:41283] regionserver.HRegionServer(1948): logDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,41283,1690229481649 2023-07-24 20:11:21,754 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:46693-0x10198f4f3ce001d, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38597,1690229472335 2023-07-24 20:11:21,753 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34773-0x10198f4f3ce001f, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38597,1690229472335 2023-07-24 20:11:21,753 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:38597-0x10198f4f3ce001e, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38597,1690229472335 2023-07-24 20:11:21,754 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,37489,1690229472216] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 2 2023-07-24 20:11:21,754 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:46693-0x10198f4f3ce001d, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41283,1690229481649 2023-07-24 20:11:21,754 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34773-0x10198f4f3ce001f, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41283,1690229481649 2023-07-24 20:11:21,754 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:38597-0x10198f4f3ce001e, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41283,1690229481649 2023-07-24 20:11:21,757 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:38597-0x10198f4f3ce001e, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46693,1690229472273 2023-07-24 20:11:21,757 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:46693-0x10198f4f3ce001d, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46693,1690229472273 2023-07-24 20:11:21,757 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:34773-0x10198f4f3ce001f, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46693,1690229472273 2023-07-24 20:11:21,757 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,37489,1690229472216] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(792): Updated with servers: 4 2023-07-24 20:11:21,764 DEBUG [RS:3;jenkins-hbase4:41283] zookeeper.ZKUtil(162): regionserver:41283-0x10198f4f3ce0028, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,34773,1690229472389 2023-07-24 20:11:21,765 DEBUG [RS:3;jenkins-hbase4:41283] zookeeper.ZKUtil(162): regionserver:41283-0x10198f4f3ce0028, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38597,1690229472335 2023-07-24 20:11:21,765 DEBUG [RS:3;jenkins-hbase4:41283] zookeeper.ZKUtil(162): regionserver:41283-0x10198f4f3ce0028, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41283,1690229481649 2023-07-24 20:11:21,766 DEBUG [RS:3;jenkins-hbase4:41283] zookeeper.ZKUtil(162): regionserver:41283-0x10198f4f3ce0028, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46693,1690229472273 2023-07-24 20:11:21,767 DEBUG [RS:3;jenkins-hbase4:41283] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-24 20:11:21,767 INFO [RS:3;jenkins-hbase4:41283] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-24 20:11:21,769 INFO [RS:3;jenkins-hbase4:41283] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-24 20:11:21,770 INFO [RS:3;jenkins-hbase4:41283] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-24 20:11:21,770 INFO [RS:3;jenkins-hbase4:41283] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:21,770 INFO [RS:3;jenkins-hbase4:41283] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-24 20:11:21,772 INFO [RS:3;jenkins-hbase4:41283] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:21,772 DEBUG [RS:3;jenkins-hbase4:41283] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:21,772 DEBUG [RS:3;jenkins-hbase4:41283] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:21,773 DEBUG [RS:3;jenkins-hbase4:41283] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:21,773 DEBUG [RS:3;jenkins-hbase4:41283] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:21,773 DEBUG [RS:3;jenkins-hbase4:41283] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:21,773 DEBUG [RS:3;jenkins-hbase4:41283] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-24 20:11:21,773 DEBUG [RS:3;jenkins-hbase4:41283] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:21,773 DEBUG [RS:3;jenkins-hbase4:41283] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:21,773 DEBUG [RS:3;jenkins-hbase4:41283] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:21,773 DEBUG [RS:3;jenkins-hbase4:41283] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:21,773 INFO [RS:3;jenkins-hbase4:41283] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:21,775 INFO [RS:3;jenkins-hbase4:41283] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:21,775 INFO [RS:3;jenkins-hbase4:41283] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:21,795 INFO [RS:3;jenkins-hbase4:41283] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-24 20:11:21,795 INFO [RS:3;jenkins-hbase4:41283] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,41283,1690229481649-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:21,818 INFO [RS:3;jenkins-hbase4:41283] regionserver.Replication(203): jenkins-hbase4.apache.org,41283,1690229481649 started 2023-07-24 20:11:21,819 INFO [RS:3;jenkins-hbase4:41283] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,41283,1690229481649, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:41283, sessionid=0x10198f4f3ce0028 2023-07-24 20:11:21,819 DEBUG [RS:3;jenkins-hbase4:41283] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-24 20:11:21,819 DEBUG [RS:3;jenkins-hbase4:41283] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,41283,1690229481649 2023-07-24 20:11:21,819 DEBUG [RS:3;jenkins-hbase4:41283] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41283,1690229481649' 2023-07-24 20:11:21,819 DEBUG [RS:3;jenkins-hbase4:41283] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 20:11:21,819 DEBUG [RS:3;jenkins-hbase4:41283] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 20:11:21,820 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-24 20:11:21,820 DEBUG [RS:3;jenkins-hbase4:41283] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-24 20:11:21,820 DEBUG [RS:3;jenkins-hbase4:41283] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-24 20:11:21,820 DEBUG [RS:3;jenkins-hbase4:41283] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,41283,1690229481649 2023-07-24 20:11:21,820 DEBUG [RS:3;jenkins-hbase4:41283] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,41283,1690229481649' 2023-07-24 20:11:21,820 DEBUG [RS:3;jenkins-hbase4:41283] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-24 20:11:21,821 DEBUG [RS:3;jenkins-hbase4:41283] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-24 20:11:21,821 DEBUG [RS:3;jenkins-hbase4:41283] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-24 20:11:21,821 INFO [RS:3;jenkins-hbase4:41283] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-24 20:11:21,821 INFO [RS:3;jenkins-hbase4:41283] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-24 20:11:21,822 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:11:21,823 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:11:21,824 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-24 20:11:21,826 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-24 20:11:21,828 DEBUG [hconnection-0x474c0116-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-24 20:11:21,829 INFO [RS-EventLoopGroup-14-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51198, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-24 20:11:21,837 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:11:21,837 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:11:21,839 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:37489] to rsgroup master 2023-07-24 20:11:21,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:37489 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-24 20:11:21,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] ipc.CallRunner(144): callId: 25 service: MasterService methodName: ExecMasterService size: 118 connection: 172.31.14.131:58688 deadline: 1690230681839, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:37489 is either offline or it does not exist. 2023-07-24 20:11:21,840 WARN [Listener at localhost/44145] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:37489 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.GeneratedConstructorAccessor62.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.afterMethod(TestRSGroupsBasics.java:82) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:37489 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-24 20:11:21,842 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:11:21,843 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:11:21,843 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:11:21,843 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:34773, jenkins-hbase4.apache.org:38597, jenkins-hbase4.apache.org:41283, jenkins-hbase4.apache.org:46693], Tables:[hbase:meta, hbase:quota, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-24 20:11:21,845 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-24 20:11:21,845 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-24 20:11:21,898 INFO [Listener at localhost/44145] hbase.ResourceChecker(175): after: rsgroup.TestRSGroupsBasics#testRSGroupsWithHBaseQuota Thread=557 (was 515) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38597 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: qtp1482583928-1649 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-14-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-1903532105-172.31.14.131-1690229433619:blk_1073741896_1072, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.8@localhost:37609 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1251407665-1998 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38597 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: PacketResponder: BP-1903532105-172.31.14.131-1690229433619:blk_1073741896_1072, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1883692998_17 at /127.0.0.1:44622 [Receiving block BP-1903532105-172.31.14.131-1690229433619:blk_1073741894_1070] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp488839288-1750 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-13 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-16-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait0(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:182) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:302) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:366) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1965004554-1705 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62881@0x0bb4fccf sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$92/589794509.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.metaPriority.FPBQ.Fifo.handler=0,queue=0,port=41283 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RS-EventLoopGroup-14-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=38597 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: qtp488839288-1749 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp277151760-1739 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-11 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp488839288-1744 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$$Lambda$75/1893131221.run(Unknown Source) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1482583928-1645 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1965004554-1707 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62881@0x4700ebae-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RS-EventLoopGroup-10-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1922791765_17 at /127.0.0.1:60348 [Receiving block BP-1903532105-172.31.14.131-1690229433619:blk_1073741892_1068] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1922791765_17 at /127.0.0.1:49980 [Receiving block BP-1903532105-172.31.14.131-1690229433619:blk_1073741892_1068] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1965004554-1710 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1883692998_17 at /127.0.0.1:60376 [Receiving block BP-1903532105-172.31.14.131-1690229433619:blk_1073741896_1072] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp66441331-1674-acceptor-0@28d48e48-ServerConnector@237e24d4{HTTP/1.1, (http/1.1)}{0.0.0.0:43467} sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:421) sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:249) org.apache.hbase.thirdparty.org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:388) org.apache.hbase.thirdparty.org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:704) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp488839288-1751 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (539191006) connection to localhost/127.0.0.1:37609 from jenkins.hfs.9 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: qtp277151760-1736 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: IPC Client (539191006) connection to localhost/127.0.0.1:37609 from jenkins.hfs.10 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-963922113_17 at /127.0.0.1:44612 [Receiving block BP-1903532105-172.31.14.131-1690229433619:blk_1073741893_1069] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp66441331-1678 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp66441331-1673 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$$Lambda$75/1893131221.run(Unknown Source) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1965004554-1708 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62881@0x5e38523b-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RpcServer.metaPriority.FPBQ.Fifo.handler=0,queue=0,port=46693 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=38597 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: qtp488839288-1745 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$$Lambda$75/1893131221.run(Unknown Source) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62881@0x45261108-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: Session-HouseKeeper-12d761c2-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-13-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1965004554-1704-acceptor-0@ca38a46-ServerConnector@34284301{HTTP/1.1, (http/1.1)}{0.0.0.0:40735} sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:421) sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:249) org.apache.hbase.thirdparty.org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:388) org.apache.hbase.thirdparty.org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:704) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1482583928-1642 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$$Lambda$75/1893131221.run(Unknown Source) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=46693 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: hconnection-0x4e07331e-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=34773 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62881@0x0621c7c0-SendThread(127.0.0.1:62881) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: qtp1482583928-1643-acceptor-0@22d045c2-ServerConnector@49f3f2e8{HTTP/1.1, (http/1.1)}{0.0.0.0:43499} sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:421) sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:249) org.apache.hbase.thirdparty.org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:388) org.apache.hbase.thirdparty.org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:704) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp66441331-1679 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-12 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=34773 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RS-EventLoopGroup-15-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.10@localhost:37609 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:1;jenkins-hbase4:38597-longCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) org.apache.hadoop.hbase.util.StealJobQueue.take(StealJobQueue.java:101) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-1903532105-172.31.14.131-1690229433619:blk_1073741894_1070, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4e07331e-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4e07331e-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: jenkins-hbase4:37489 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) org.apache.hadoop.hbase.master.assignment.AssignmentManager.waitOnAssignQueue(AssignmentManager.java:2102) org.apache.hadoop.hbase.master.assignment.AssignmentManager.processAssignQueue(AssignmentManager.java:2124) org.apache.hadoop.hbase.master.assignment.AssignmentManager.access$600(AssignmentManager.java:104) org.apache.hadoop.hbase.master.assignment.AssignmentManager$1.run(AssignmentManager.java:2064) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=46693 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: PacketResponder: BP-1903532105-172.31.14.131-1690229433619:blk_1073741896_1072, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-16-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait0(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:182) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:302) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:366) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1965004554-1703 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$$Lambda$75/1893131221.run(Unknown Source) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp277151760-1737 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=46693 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=41283 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: M:0;jenkins-hbase4:37489 java.lang.Object.wait(Native Method) org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:81) org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:64) org.apache.hadoop.hbase.master.HMaster.waitForMasterActive(HMaster.java:634) org.apache.hadoop.hbase.regionserver.HRegionServer.initializeZooKeeper(HRegionServer.java:957) org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:904) org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1006) org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:541) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-14 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:68) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:879) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=34773 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62881@0x1c312feb sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$92/589794509.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-15-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-1903532105-172.31.14.131-1690229433619:blk_1073741895_1071, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62881@0x4700ebae-SendThread(127.0.0.1:62881) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62881@0x68001b57-SendThread(127.0.0.1:62881) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=0,queue=0,port=46693 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RS:0;jenkins-hbase4:46693-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1251407665-1995 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-12-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-963922113_17 at /127.0.0.1:60358 [Receiving block BP-1903532105-172.31.14.131-1690229433619:blk_1073741893_1069] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-1903532105-172.31.14.131-1690229433619:blk_1073741893_1069, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: AsyncFSWAL-0-hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4-prefix:jenkins-hbase4.apache.org,46693,1690229472273.meta sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62881@0x47c01838-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RS:1;jenkins-hbase4:38597 java.lang.Object.wait(Native Method) org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:81) org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:64) org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1092) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:175) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:123) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:159) java.security.AccessController.doPrivileged(Native Method) javax.security.auth.Subject.doAs(Subject.java:360) org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1873) org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:319) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:156) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62881@0x1c312feb-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,37489,1690229472216 java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread.run(RSGroupInfoManagerImpl.java:797) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41283 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: PacketResponder: BP-1903532105-172.31.14.131-1690229433619:blk_1073741895_1071, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-963922113_17 at /127.0.0.1:49984 [Receiving block BP-1903532105-172.31.14.131-1690229433619:blk_1073741893_1069] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-1903532105-172.31.14.131-1690229433619:blk_1073741892_1068, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4e07331e-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp277151760-1735 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x474c0116-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-9-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-11-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=41283 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RS-EventLoopGroup-16-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait0(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:182) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:302) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:366) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: jenkins-hbase4:46693Replication Statistics #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-13-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-1903532105-172.31.14.131-1690229433619:blk_1073741894_1070, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1066534066_17 at /127.0.0.1:60372 [Receiving block BP-1903532105-172.31.14.131-1690229433619:blk_1073741895_1071] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37489 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: qtp1251407665-1991 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$$Lambda$75/1893131221.run(Unknown Source) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1482583928-1646 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62881@0x1c312feb-SendThread(127.0.0.1:62881) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: jenkins-hbase4:38597Replication Statistics #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/cluster_82e812c0-1679-84b5-414c-93cfc125cc3f/dfs/data/data4/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-13-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-17-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait0(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:182) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWait(EpollEventLoop.java:302) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:366) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=34773 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RS-EventLoopGroup-14-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1251407665-1992-acceptor-0@21fe6351-ServerConnector@76b99374{HTTP/1.1, (http/1.1)}{0.0.0.0:41237} sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:421) sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:249) org.apache.hbase.thirdparty.org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:388) org.apache.hbase.thirdparty.org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:704) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.metaPriority.FPBQ.Fifo.handler=0,queue=0,port=34773 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1922791765_17 at /127.0.0.1:44600 [Receiving block BP-1903532105-172.31.14.131-1690229433619:blk_1073741892_1068] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62881@0x0621c7c0-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: qtp1251407665-1996 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1251407665-1994 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.5@localhost:37609 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=46693 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RpcServer.metaPriority.FPBQ.Fifo.handler=0,queue=0,port=38597 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1066534066_17 at /127.0.0.1:44616 [Receiving block BP-1903532105-172.31.14.131-1690229433619:blk_1073741895_1071] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: jenkins-hbase4:41283Replication Statistics #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=0,queue=0,port=34773 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RS-EventLoopGroup-17-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/cluster_82e812c0-1679-84b5-414c-93cfc125cc3f/dfs/data/data3/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1482583928-1647 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp277151760-1733 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$$Lambda$75/1893131221.run(Unknown Source) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1482583928-1644 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp488839288-1746 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$$Lambda$75/1893131221.run(Unknown Source) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-15-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-1903532105-172.31.14.131-1690229433619:blk_1073741892_1068, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Session-HouseKeeper-6bdc114e-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Session-HouseKeeper-410aafa0-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=46693 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: qtp277151760-1734-acceptor-0@44910cdc-ServerConnector@24f0b740{HTTP/1.1, (http/1.1)}{0.0.0.0:34127} sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:421) sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:249) org.apache.hbase.thirdparty.org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:388) org.apache.hbase.thirdparty.org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:704) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38597 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=41283 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41283 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=0,queue=0,port=41283 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RS:2;jenkins-hbase4:34773-longCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) org.apache.hadoop.hbase.util.StealJobQueue.take(StealJobQueue.java:101) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1883692998_17 at /127.0.0.1:50010 [Receiving block BP-1903532105-172.31.14.131-1690229433619:blk_1073741896_1072] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: LeaseRenewer:jenkins.hfs.6@localhost:37609 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=37489 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1883692998_17 at /127.0.0.1:50008 [Receiving block BP-1903532105-172.31.14.131-1690229433619:blk_1073741894_1070] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62881@0x47c01838 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$92/589794509.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: AsyncFSWAL-0-hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData-prefix:jenkins-hbase4.apache.org,37489,1690229472216 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34773 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34773 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=0,queue=0,port=38597 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,33409,1690229459778 java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread.run(RSGroupInfoManagerImpl.java:797) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1066534066_17 at /127.0.0.1:49998 [Receiving block BP-1903532105-172.31.14.131-1690229433619:blk_1073741895_1071] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:46693 java.lang.Object.wait(Native Method) org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:81) org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:64) org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1092) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:175) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:123) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:159) java.security.AccessController.doPrivileged(Native Method) javax.security.auth.Subject.doAs(Subject.java:360) org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1873) org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:319) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:156) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=41283 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: IPC Client (539191006) connection to localhost/127.0.0.1:37609 from jenkins.hfs.8 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: RS-EventLoopGroup-12-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: jenkins-hbase4:34773Replication Statistics #0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:46693-longCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) org.apache.hadoop.hbase.util.StealJobQueue.take(StealJobQueue.java:101) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Client (539191006) connection to localhost/127.0.0.1:37609 from jenkins.hfs.11 java.lang.Object.wait(Native Method) org.apache.hadoop.ipc.Client$Connection.waitForWork(Client.java:1035) org.apache.hadoop.ipc.Client$Connection.run(Client.java:1079) Potentially hanging thread: Session-HouseKeeper-71eab039-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp66441331-1675 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp277151760-1738 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62881@0x5e38523b sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$92/589794509.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1690229472734 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) org.apache.hadoop.hbase.util.StealJobQueue.take(StealJobQueue.java:101) org.apache.hadoop.hbase.master.cleaner.HFileCleaner.consumerLoop(HFileCleaner.java:267) org.apache.hadoop.hbase.master.cleaner.HFileCleaner$1.run(HFileCleaner.java:236) Potentially hanging thread: PacketResponder: BP-1903532105-172.31.14.131-1690229433619:blk_1073741893_1069, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62881@0x68001b57-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1883692998_17 at /127.0.0.1:44630 [Receiving block BP-1903532105-172.31.14.131-1690229433619:blk_1073741896_1072] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62881@0x68001b57 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$92/589794509.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=37489 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34773 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37489 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37489 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62881@0x4700ebae sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$92/589794509.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=38597 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: qtp1251407665-1997 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Session-HouseKeeper-6ed4cfcf-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1965004554-1706 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:2;jenkins-hbase4:34773 java.lang.Object.wait(Native Method) org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:81) org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:64) org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1092) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:175) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:123) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:159) java.security.AccessController.doPrivileged(Native Method) javax.security.auth.Subject.doAs(Subject.java:360) org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1873) org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:319) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:156) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: AsyncFSWAL-0-hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4-prefix:jenkins-hbase4.apache.org,38597,1690229472335 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-1903532105-172.31.14.131-1690229433619:blk_1073741892_1068, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=46693 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62881@0x0bb4fccf-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_703487477_17 at /127.0.0.1:37552 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=41283 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: AsyncFSWAL-0-hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4-prefix:jenkins-hbase4.apache.org,46693,1690229472273 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62881@0x45261108-SendThread(127.0.0.1:62881) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: LeaseRenewer:jenkins.hfs.9@localhost:37609 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp488839288-1747 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:183) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:190) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:606) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:543) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:362) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:186) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:137) org.apache.hbase.thirdparty.org.eclipse.jetty.io.ManagedSelector$$Lambda$75/1893131221.run(Unknown Source) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=0,queue=0,port=37489 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: LeaseRenewer:jenkins.hfs.7@localhost:37609 java.lang.Thread.sleep(Native Method) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.run(LeaseRenewer.java:411) org.apache.hadoop.hdfs.client.impl.LeaseRenewer.access$600(LeaseRenewer.java:76) org.apache.hadoop.hdfs.client.impl.LeaseRenewer$1.run(LeaseRenewer.java:307) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62881@0x0621c7c0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$92/589794509.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1482583928-1648 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=46693 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: PacketResponder: BP-1903532105-172.31.14.131-1690229433619:blk_1073741893_1069, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62881@0x0bb4fccf-SendThread(127.0.0.1:62881) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37489 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RS:3;jenkins-hbase4:41283 java.lang.Object.wait(Native Method) org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:81) org.apache.hadoop.hbase.util.Sleeper.sleep(Sleeper.java:64) org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1092) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:175) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:123) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:159) java.security.AccessController.doPrivileged(Native Method) javax.security.auth.Subject.doAs(Subject.java:360) org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1873) org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:319) org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:156) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1883692998_17 at /127.0.0.1:60360 [Receiving block BP-1903532105-172.31.14.131-1690229433619:blk_1073741894_1070] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp66441331-1680 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-10-1 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.metaPriority.FPBQ.Fifo.handler=0,queue=0,port=37489 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=46693 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41283 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: PacketResponder: BP-1903532105-172.31.14.131-1690229433619:blk_1073741895_1071, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=1,queue=0,port=38597 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: hconnection-0x474c0116-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38597 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: RpcServer.replication.FPBQ.Fifo.handler=2,queue=0,port=37489 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) java.util.concurrent.Semaphore.acquire(Semaphore.java:312) org.apache.hadoop.hbase.ipc.FastPathBalancedQueueRpcExecutor$FastPathHandler.getCallRunner(FastPathBalancedQueueRpcExecutor.java:105) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: qtp66441331-1677 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=34773 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.getCallRunner(RpcExecutor.java:340) org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) Potentially hanging thread: region-location-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: AsyncFSWAL-0-hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4-prefix:jenkins-hbase4.apache.org,34773,1690229472389 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp66441331-1676 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:3;jenkins-hbase4:41283-longCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) org.apache.hadoop.hbase.util.StealJobQueue.take(StealJobQueue.java:101) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp277151760-1740 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1251407665-1993 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp1965004554-1709 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) org.apache.hbase.thirdparty.org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:382) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.idleJobPoll(QueuedThreadPool.java:974) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1018) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62881@0x47c01838-SendThread(127.0.0.1:62881) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-15 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62881@0x5e38523b-SendThread(127.0.0.1:62881) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:62881@0x45261108 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$92/589794509.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-1903532105-172.31.14.131-1690229433619:blk_1073741894_1070, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-16 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: qtp488839288-1748-acceptor-0@7d0bec28-ServerConnector@13e4f5ac{HTTP/1.1, (http/1.1)}{0.0.0.0:44415} sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:421) sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:249) org.apache.hbase.thirdparty.org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:388) org.apache.hbase.thirdparty.org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:704) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) org.apache.hbase.thirdparty.org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1690229472739 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) org.apache.hadoop.hbase.master.cleaner.HFileCleaner.consumerLoop(HFileCleaner.java:267) org.apache.hadoop.hbase.master.cleaner.HFileCleaner$2.run(HFileCleaner.java:251) - Thread LEAK? -, OpenFileDescriptor=872 (was 791) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=370 (was 408), ProcessCount=174 (was 176), AvailableMemoryMB=7768 (was 5716) - AvailableMemoryMB LEAK? - 2023-07-24 20:11:21,902 WARN [Listener at localhost/44145] hbase.ResourceChecker(130): Thread=557 is superior to 500 2023-07-24 20:11:21,925 INFO [RS:3;jenkins-hbase4:41283] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C41283%2C1690229481649, suffix=, logDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,41283,1690229481649, archiveDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/oldWALs, maxLogs=32 2023-07-24 20:11:21,925 INFO [Listener at localhost/44145] hbase.ResourceChecker(147): before: rsgroup.TestRSGroupsBasics#testClearDeadServers Thread=557, OpenFileDescriptor=872, MaxFileDescriptor=60000, SystemLoadAverage=370, ProcessCount=174, AvailableMemoryMB=7762 2023-07-24 20:11:21,925 WARN [Listener at localhost/44145] hbase.ResourceChecker(130): Thread=557 is superior to 500 2023-07-24 20:11:21,925 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase(132): testClearDeadServers 2023-07-24 20:11:21,937 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:11:21,938 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:11:21,939 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-24 20:11:21,939 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-24 20:11:21,939 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-24 20:11:21,941 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-24 20:11:21,941 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-24 20:11:21,942 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-24 20:11:21,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:11:21,947 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-24 20:11:21,955 DEBUG [RS-EventLoopGroup-17-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK] 2023-07-24 20:11:21,955 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-24 20:11:21,955 DEBUG [RS-EventLoopGroup-17-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK] 2023-07-24 20:11:21,956 DEBUG [RS-EventLoopGroup-17-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK] 2023-07-24 20:11:21,959 INFO [RS:3;jenkins-hbase4:41283] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,41283,1690229481649/jenkins-hbase4.apache.org%2C41283%2C1690229481649.1690229481926 2023-07-24 20:11:21,959 DEBUG [RS:3;jenkins-hbase4:41283] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37501,DS-a5b4792c-4be5-4b56-947a-d97b275970c9,DISK], DatanodeInfoWithStorage[127.0.0.1:38567,DS-062e8295-abc1-45d4-88c5-80d70c08380a,DISK], DatanodeInfoWithStorage[127.0.0.1:34425,DS-d723b047-8a44-4a99-b3c7-f9704403895f,DISK]] 2023-07-24 20:11:21,960 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase(152): Restoring servers: 0 2023-07-24 20:11:21,961 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-24 20:11:21,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:11:21,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:11:21,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-24 20:11:21,966 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-24 20:11:21,968 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:11:21,968 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:11:21,970 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:37489] to rsgroup master 2023-07-24 20:11:21,970 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:37489 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-24 20:11:21,971 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] ipc.CallRunner(144): callId: 53 service: MasterService methodName: ExecMasterService size: 118 connection: 172.31.14.131:58688 deadline: 1690230681970, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:37489 is either offline or it does not exist. 2023-07-24 20:11:21,971 WARN [Listener at localhost/44145] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:37489 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.GeneratedConstructorAccessor62.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.setUpBeforeMethod(TestRSGroupsBase.java:136) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.beforeMethod(TestRSGroupsBasics.java:77) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:37489 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-24 20:11:21,972 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:11:21,973 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:11:21,973 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:11:21,973 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:34773, jenkins-hbase4.apache.org:38597, jenkins-hbase4.apache.org:41283, jenkins-hbase4.apache.org:46693], Tables:[hbase:meta, hbase:quota, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-24 20:11:21,974 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-24 20:11:21,974 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-24 20:11:21,974 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBasics(214): testClearDeadServers 2023-07-24 20:11:21,975 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-24 20:11:21,975 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-24 20:11:21,975 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup Group_testClearDeadServers_1855695895 2023-07-24 20:11:21,977 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/Group_testClearDeadServers_1855695895 2023-07-24 20:11:21,980 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:11:21,980 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:11:21,980 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 6 2023-07-24 20:11:21,982 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-24 20:11:21,984 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:11:21,984 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:11:21,987 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:41283, jenkins-hbase4.apache.org:34773, jenkins-hbase4.apache.org:38597] to rsgroup Group_testClearDeadServers_1855695895 2023-07-24 20:11:21,988 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:11:21,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/Group_testClearDeadServers_1855695895 2023-07-24 20:11:21,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:11:21,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 6 2023-07-24 20:11:21,991 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminServer(238): Moving server region 4fa2a21e5a49480a149a055b0fc39515, which do not belong to RSGroup Group_testClearDeadServers_1855695895 2023-07-24 20:11:21,991 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] procedure2.ProcedureExecutor(1029): Stored pid=133, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=hbase:quota, region=4fa2a21e5a49480a149a055b0fc39515, REOPEN/MOVE 2023-07-24 20:11:21,992 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminServer(238): Moving server region 324410cf2a29c2dca1a19e8570799d47, which do not belong to RSGroup Group_testClearDeadServers_1855695895 2023-07-24 20:11:21,992 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=133, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=hbase:quota, region=4fa2a21e5a49480a149a055b0fc39515, REOPEN/MOVE 2023-07-24 20:11:21,992 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] procedure2.ProcedureExecutor(1029): Stored pid=134, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=hbase:namespace, region=324410cf2a29c2dca1a19e8570799d47, REOPEN/MOVE 2023-07-24 20:11:21,992 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=133 updating hbase:meta row=4fa2a21e5a49480a149a055b0fc39515, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,34773,1690229472389 2023-07-24 20:11:21,992 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=134, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=hbase:namespace, region=324410cf2a29c2dca1a19e8570799d47, REOPEN/MOVE 2023-07-24 20:11:21,992 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminServer(286): Moving 2 region(s) to group default, current retry=0 2023-07-24 20:11:21,992 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515.","families":{"info":[{"qualifier":"regioninfo","vlen":37,"tag":[],"timestamp":"1690229481992"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229481992"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229481992"}]},"ts":"1690229481992"} 2023-07-24 20:11:21,993 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=134 updating hbase:meta row=324410cf2a29c2dca1a19e8570799d47, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,38597,1690229472335 2023-07-24 20:11:21,993 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690229481993"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229481993"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229481993"}]},"ts":"1690229481993"} 2023-07-24 20:11:22,021 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=135, ppid=133, state=RUNNABLE; CloseRegionProcedure 4fa2a21e5a49480a149a055b0fc39515, server=jenkins-hbase4.apache.org,34773,1690229472389}] 2023-07-24 20:11:22,025 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4700ebae to 127.0.0.1:62881 2023-07-24 20:11:22,025 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=134, state=RUNNABLE; CloseRegionProcedure 324410cf2a29c2dca1a19e8570799d47, server=jenkins-hbase4.apache.org,38597,1690229472335}] 2023-07-24 20:11:22,025 DEBUG [Finalizer] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:11:22,178 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 4fa2a21e5a49480a149a055b0fc39515 2023-07-24 20:11:22,179 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 4fa2a21e5a49480a149a055b0fc39515, disabling compactions & flushes 2023-07-24 20:11:22,179 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515. 2023-07-24 20:11:22,179 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515. 2023-07-24 20:11:22,179 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515. after waiting 0 ms 2023-07-24 20:11:22,179 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515. 2023-07-24 20:11:22,180 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 324410cf2a29c2dca1a19e8570799d47 2023-07-24 20:11:22,181 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 324410cf2a29c2dca1a19e8570799d47, disabling compactions & flushes 2023-07-24 20:11:22,181 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. 2023-07-24 20:11:22,181 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. 2023-07-24 20:11:22,181 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. after waiting 0 ms 2023-07-24 20:11:22,181 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. 2023-07-24 20:11:22,187 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/quota/4fa2a21e5a49480a149a055b0fc39515/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-07-24 20:11:22,188 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/namespace/324410cf2a29c2dca1a19e8570799d47/recovered.edits/20.seqid, newMaxSeqId=20, maxSeqId=17 2023-07-24 20:11:22,188 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515. 2023-07-24 20:11:22,188 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 4fa2a21e5a49480a149a055b0fc39515: 2023-07-24 20:11:22,188 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3513): Adding 4fa2a21e5a49480a149a055b0fc39515 move to jenkins-hbase4.apache.org,46693,1690229472273 record at close sequenceid=5 2023-07-24 20:11:22,189 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. 2023-07-24 20:11:22,189 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 324410cf2a29c2dca1a19e8570799d47: 2023-07-24 20:11:22,189 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3513): Adding 324410cf2a29c2dca1a19e8570799d47 move to jenkins-hbase4.apache.org,46693,1690229472273 record at close sequenceid=18 2023-07-24 20:11:22,190 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 4fa2a21e5a49480a149a055b0fc39515 2023-07-24 20:11:22,191 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=133 updating hbase:meta row=4fa2a21e5a49480a149a055b0fc39515, regionState=CLOSED 2023-07-24 20:11:22,191 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515.","families":{"info":[{"qualifier":"regioninfo","vlen":37,"tag":[],"timestamp":"1690229482190"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229482190"}]},"ts":"1690229482190"} 2023-07-24 20:11:22,191 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 324410cf2a29c2dca1a19e8570799d47 2023-07-24 20:11:22,191 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=134 updating hbase:meta row=324410cf2a29c2dca1a19e8570799d47, regionState=CLOSED 2023-07-24 20:11:22,191 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690229482191"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690229482191"}]},"ts":"1690229482191"} 2023-07-24 20:11:22,194 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=135, resume processing ppid=133 2023-07-24 20:11:22,194 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=135, ppid=133, state=SUCCESS; CloseRegionProcedure 4fa2a21e5a49480a149a055b0fc39515, server=jenkins-hbase4.apache.org,34773,1690229472389 in 171 msec 2023-07-24 20:11:22,195 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=136, resume processing ppid=134 2023-07-24 20:11:22,195 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=134, state=SUCCESS; CloseRegionProcedure 324410cf2a29c2dca1a19e8570799d47, server=jenkins-hbase4.apache.org,38597,1690229472335 in 168 msec 2023-07-24 20:11:22,195 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=133, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:quota, region=4fa2a21e5a49480a149a055b0fc39515, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,46693,1690229472273; forceNewPlan=false, retain=false 2023-07-24 20:11:22,195 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=134, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=324410cf2a29c2dca1a19e8570799d47, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,46693,1690229472273; forceNewPlan=false, retain=false 2023-07-24 20:11:22,196 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=133 updating hbase:meta row=4fa2a21e5a49480a149a055b0fc39515, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46693,1690229472273 2023-07-24 20:11:22,196 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515.","families":{"info":[{"qualifier":"regioninfo","vlen":37,"tag":[],"timestamp":"1690229482196"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229482196"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229482196"}]},"ts":"1690229482196"} 2023-07-24 20:11:22,196 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=134 updating hbase:meta row=324410cf2a29c2dca1a19e8570799d47, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,46693,1690229472273 2023-07-24 20:11:22,197 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690229482196"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690229482196"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690229482196"}]},"ts":"1690229482196"} 2023-07-24 20:11:22,197 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=137, ppid=133, state=RUNNABLE; OpenRegionProcedure 4fa2a21e5a49480a149a055b0fc39515, server=jenkins-hbase4.apache.org,46693,1690229472273}] 2023-07-24 20:11:22,198 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=134, state=RUNNABLE; OpenRegionProcedure 324410cf2a29c2dca1a19e8570799d47, server=jenkins-hbase4.apache.org,46693,1690229472273}] 2023-07-24 20:11:22,356 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. 2023-07-24 20:11:22,356 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 324410cf2a29c2dca1a19e8570799d47, NAME => 'hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47.', STARTKEY => '', ENDKEY => ''} 2023-07-24 20:11:22,356 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace 324410cf2a29c2dca1a19e8570799d47 2023-07-24 20:11:22,356 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:11:22,356 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 324410cf2a29c2dca1a19e8570799d47 2023-07-24 20:11:22,356 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 324410cf2a29c2dca1a19e8570799d47 2023-07-24 20:11:22,358 INFO [StoreOpener-324410cf2a29c2dca1a19e8570799d47-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 324410cf2a29c2dca1a19e8570799d47 2023-07-24 20:11:22,359 DEBUG [StoreOpener-324410cf2a29c2dca1a19e8570799d47-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/namespace/324410cf2a29c2dca1a19e8570799d47/info 2023-07-24 20:11:22,359 DEBUG [StoreOpener-324410cf2a29c2dca1a19e8570799d47-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/namespace/324410cf2a29c2dca1a19e8570799d47/info 2023-07-24 20:11:22,359 INFO [StoreOpener-324410cf2a29c2dca1a19e8570799d47-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 324410cf2a29c2dca1a19e8570799d47 columnFamilyName info 2023-07-24 20:11:22,372 INFO [StoreFileOpener-info-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 81b8365c87814c2e9ac1bfab2b6bb254 2023-07-24 20:11:22,373 DEBUG [StoreOpener-324410cf2a29c2dca1a19e8570799d47-1] regionserver.HStore(539): loaded hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/namespace/324410cf2a29c2dca1a19e8570799d47/info/81b8365c87814c2e9ac1bfab2b6bb254 2023-07-24 20:11:22,373 INFO [StoreOpener-324410cf2a29c2dca1a19e8570799d47-1] regionserver.HStore(310): Store=324410cf2a29c2dca1a19e8570799d47/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:11:22,374 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/namespace/324410cf2a29c2dca1a19e8570799d47 2023-07-24 20:11:22,376 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/namespace/324410cf2a29c2dca1a19e8570799d47 2023-07-24 20:11:22,380 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 324410cf2a29c2dca1a19e8570799d47 2023-07-24 20:11:22,382 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 324410cf2a29c2dca1a19e8570799d47; next sequenceid=21; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=9483678880, jitterRate=-0.11676357686519623}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 20:11:22,382 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 324410cf2a29c2dca1a19e8570799d47: 2023-07-24 20:11:22,384 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47., pid=138, masterSystemTime=1690229482349 2023-07-24 20:11:22,386 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. 2023-07-24 20:11:22,386 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. 2023-07-24 20:11:22,386 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515. 2023-07-24 20:11:22,386 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 4fa2a21e5a49480a149a055b0fc39515, NAME => 'hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515.', STARTKEY => '', ENDKEY => ''} 2023-07-24 20:11:22,386 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table quota 4fa2a21e5a49480a149a055b0fc39515 2023-07-24 20:11:22,386 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 20:11:22,386 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 4fa2a21e5a49480a149a055b0fc39515 2023-07-24 20:11:22,386 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 4fa2a21e5a49480a149a055b0fc39515 2023-07-24 20:11:22,387 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=134 updating hbase:meta row=324410cf2a29c2dca1a19e8570799d47, regionState=OPEN, openSeqNum=21, regionLocation=jenkins-hbase4.apache.org,46693,1690229472273 2023-07-24 20:11:22,387 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690229482387"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690229482387"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690229482387"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690229482387"}]},"ts":"1690229482387"} 2023-07-24 20:11:22,391 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=134 2023-07-24 20:11:22,391 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=134, state=SUCCESS; OpenRegionProcedure 324410cf2a29c2dca1a19e8570799d47, server=jenkins-hbase4.apache.org,46693,1690229472273 in 191 msec 2023-07-24 20:11:22,393 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=134, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=324410cf2a29c2dca1a19e8570799d47, REOPEN/MOVE in 399 msec 2023-07-24 20:11:22,403 INFO [StoreOpener-4fa2a21e5a49480a149a055b0fc39515-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family q of region 4fa2a21e5a49480a149a055b0fc39515 2023-07-24 20:11:22,404 DEBUG [StoreOpener-4fa2a21e5a49480a149a055b0fc39515-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/quota/4fa2a21e5a49480a149a055b0fc39515/q 2023-07-24 20:11:22,404 DEBUG [StoreOpener-4fa2a21e5a49480a149a055b0fc39515-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/quota/4fa2a21e5a49480a149a055b0fc39515/q 2023-07-24 20:11:22,405 INFO [StoreOpener-4fa2a21e5a49480a149a055b0fc39515-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4fa2a21e5a49480a149a055b0fc39515 columnFamilyName q 2023-07-24 20:11:22,405 INFO [StoreOpener-4fa2a21e5a49480a149a055b0fc39515-1] regionserver.HStore(310): Store=4fa2a21e5a49480a149a055b0fc39515/q, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:11:22,406 INFO [StoreOpener-4fa2a21e5a49480a149a055b0fc39515-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family u of region 4fa2a21e5a49480a149a055b0fc39515 2023-07-24 20:11:22,407 DEBUG [StoreOpener-4fa2a21e5a49480a149a055b0fc39515-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/quota/4fa2a21e5a49480a149a055b0fc39515/u 2023-07-24 20:11:22,407 DEBUG [StoreOpener-4fa2a21e5a49480a149a055b0fc39515-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/quota/4fa2a21e5a49480a149a055b0fc39515/u 2023-07-24 20:11:22,407 INFO [StoreOpener-4fa2a21e5a49480a149a055b0fc39515-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4fa2a21e5a49480a149a055b0fc39515 columnFamilyName u 2023-07-24 20:11:22,408 INFO [StoreOpener-4fa2a21e5a49480a149a055b0fc39515-1] regionserver.HStore(310): Store=4fa2a21e5a49480a149a055b0fc39515/u, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 20:11:22,409 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/quota/4fa2a21e5a49480a149a055b0fc39515 2023-07-24 20:11:22,412 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/quota/4fa2a21e5a49480a149a055b0fc39515 2023-07-24 20:11:22,415 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:quota descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 20:11:22,417 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 4fa2a21e5a49480a149a055b0fc39515 2023-07-24 20:11:22,419 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 4fa2a21e5a49480a149a055b0fc39515; next sequenceid=8; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=11040830240, jitterRate=0.028257444500923157}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 20:11:22,419 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 4fa2a21e5a49480a149a055b0fc39515: 2023-07-24 20:11:22,422 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515., pid=137, masterSystemTime=1690229482349 2023-07-24 20:11:22,424 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515. 2023-07-24 20:11:22,424 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515. 2023-07-24 20:11:22,424 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=133 updating hbase:meta row=4fa2a21e5a49480a149a055b0fc39515, regionState=OPEN, openSeqNum=8, regionLocation=jenkins-hbase4.apache.org,46693,1690229472273 2023-07-24 20:11:22,425 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515.","families":{"info":[{"qualifier":"regioninfo","vlen":37,"tag":[],"timestamp":"1690229482424"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690229482424"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690229482424"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690229482424"}]},"ts":"1690229482424"} 2023-07-24 20:11:22,429 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=137, resume processing ppid=133 2023-07-24 20:11:22,429 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=133, state=SUCCESS; OpenRegionProcedure 4fa2a21e5a49480a149a055b0fc39515, server=jenkins-hbase4.apache.org,46693,1690229472273 in 229 msec 2023-07-24 20:11:22,431 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=133, state=SUCCESS; TransitRegionStateProcedure table=hbase:quota, region=4fa2a21e5a49480a149a055b0fc39515, REOPEN/MOVE in 438 msec 2023-07-24 20:11:22,993 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] procedure.ProcedureSyncWait(216): waitFor pid=133 2023-07-24 20:11:22,993 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminServer(261): All regions from [jenkins-hbase4.apache.org,34773,1690229472389, jenkins-hbase4.apache.org,38597,1690229472335, jenkins-hbase4.apache.org,41283,1690229481649] are moved back to default 2023-07-24 20:11:22,993 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminServer(438): Move servers done: default => Group_testClearDeadServers_1855695895 2023-07-24 20:11:22,993 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-24 20:11:22,997 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:11:22,997 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:11:23,000 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=Group_testClearDeadServers_1855695895 2023-07-24 20:11:23,000 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-24 20:11:23,001 DEBUG [Listener at localhost/44145] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-24 20:11:23,002 INFO [RS-EventLoopGroup-16-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51076, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-24 20:11:23,002 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34773] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,34773,1690229472389' ***** 2023-07-24 20:11:23,003 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34773] regionserver.HRegionServer(2311): STOPPED: Called by admin client hconnection-0x60f3df78 2023-07-24 20:11:23,003 INFO [RS:2;jenkins-hbase4:34773] regionserver.HRegionServer(1109): Stopping infoServer 2023-07-24 20:11:23,006 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:11:23,006 INFO [RS:2;jenkins-hbase4:34773] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.w.WebAppContext@514b4e4e{regionserver,/,null,STOPPED}{file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver} 2023-07-24 20:11:23,007 INFO [RS:2;jenkins-hbase4:34773] server.AbstractConnector(383): Stopped ServerConnector@24f0b740{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-24 20:11:23,007 INFO [RS:2;jenkins-hbase4:34773] session.HouseKeeper(149): node0 Stopped scavenging 2023-07-24 20:11:23,010 INFO [RS:2;jenkins-hbase4:34773] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@409405c{static,/static,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/static/,STOPPED} 2023-07-24 20:11:23,011 INFO [RS:2;jenkins-hbase4:34773] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@5405ec89{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/hadoop.log.dir/,STOPPED} 2023-07-24 20:11:23,012 INFO [RS:2;jenkins-hbase4:34773] regionserver.HeapMemoryManager(220): Stopping 2023-07-24 20:11:23,012 INFO [RS:2;jenkins-hbase4:34773] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-24 20:11:23,012 INFO [RS:2;jenkins-hbase4:34773] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-24 20:11:23,012 INFO [RS:2;jenkins-hbase4:34773] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,34773,1690229472389 2023-07-24 20:11:23,012 DEBUG [RS:2;jenkins-hbase4:34773] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x68001b57 to 127.0.0.1:62881 2023-07-24 20:11:23,012 DEBUG [RS:2;jenkins-hbase4:34773] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:11:23,013 INFO [RS:2;jenkins-hbase4:34773] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,34773,1690229472389; all regions closed. 2023-07-24 20:11:23,013 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-24 20:11:23,027 DEBUG [RS:2;jenkins-hbase4:34773] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/oldWALs 2023-07-24 20:11:23,027 INFO [RS:2;jenkins-hbase4:34773] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C34773%2C1690229472389:(num 1690229472977) 2023-07-24 20:11:23,027 DEBUG [RS:2;jenkins-hbase4:34773] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:11:23,027 INFO [RS:2;jenkins-hbase4:34773] regionserver.LeaseManager(133): Closed leases 2023-07-24 20:11:23,028 INFO [RS:2;jenkins-hbase4:34773] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-24 20:11:23,028 INFO [RS:2;jenkins-hbase4:34773] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-24 20:11:23,028 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-24 20:11:23,028 INFO [RS:2;jenkins-hbase4:34773] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-24 20:11:23,028 INFO [RS:2;jenkins-hbase4:34773] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-24 20:11:23,030 INFO [RS:2;jenkins-hbase4:34773] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:34773 2023-07-24 20:11:23,032 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:38597-0x10198f4f3ce001e, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34773,1690229472389 2023-07-24 20:11:23,032 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:41283-0x10198f4f3ce0028, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34773,1690229472389 2023-07-24 20:11:23,033 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:11:23,033 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34773-0x10198f4f3ce001f, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34773,1690229472389 2023-07-24 20:11:23,033 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34773-0x10198f4f3ce001f, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:11:23,033 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:41283-0x10198f4f3ce0028, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:11:23,033 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:38597-0x10198f4f3ce001e, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:11:23,033 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:46693-0x10198f4f3ce001d, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,34773,1690229472389 2023-07-24 20:11:23,033 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:46693-0x10198f4f3ce001d, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:11:23,034 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,34773,1690229472389] 2023-07-24 20:11:23,034 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,34773,1690229472389; numProcessing=1 2023-07-24 20:11:23,035 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:41283-0x10198f4f3ce0028, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38597,1690229472335 2023-07-24 20:11:23,035 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,34773,1690229472389 already deleted, retry=false 2023-07-24 20:11:23,035 INFO [RegionServerTracker-0] master.ServerManager(568): Processing expiration of jenkins-hbase4.apache.org,34773,1690229472389 on jenkins-hbase4.apache.org,37489,1690229472216 2023-07-24 20:11:23,036 DEBUG [RegionServerTracker-0] procedure2.ProcedureExecutor(1029): Stored pid=139, state=RUNNABLE:SERVER_CRASH_START; ServerCrashProcedure jenkins-hbase4.apache.org,34773,1690229472389, splitWal=true, meta=false 2023-07-24 20:11:23,036 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:46693-0x10198f4f3ce001d, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38597,1690229472335 2023-07-24 20:11:23,037 INFO [RegionServerTracker-0] assignment.AssignmentManager(1734): Scheduled ServerCrashProcedure pid=139 for jenkins-hbase4.apache.org,34773,1690229472389 (carryingMeta=false) jenkins-hbase4.apache.org,34773,1690229472389/CRASHED/regionCount=0/lock=java.util.concurrent.locks.ReentrantReadWriteLock@38ee3de6[Write locks = 1, Read locks = 0], oldState=ONLINE. 2023-07-24 20:11:23,037 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,37489,1690229472216] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(787): Updating default servers. 2023-07-24 20:11:23,039 INFO [PEWorker-3] procedure.ServerCrashProcedure(161): Start pid=139, state=RUNNABLE:SERVER_CRASH_START, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,34773,1690229472389, splitWal=true, meta=false 2023-07-24 20:11:23,040 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:38597-0x10198f4f3ce001e, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38597,1690229472335 2023-07-24 20:11:23,040 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:41283-0x10198f4f3ce0028, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41283,1690229481649 2023-07-24 20:11:23,040 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:46693-0x10198f4f3ce001d, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41283,1690229481649 2023-07-24 20:11:23,041 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:38597-0x10198f4f3ce001e, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41283,1690229481649 2023-07-24 20:11:23,041 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:41283-0x10198f4f3ce0028, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46693,1690229472273 2023-07-24 20:11:23,041 INFO [PEWorker-3] procedure.ServerCrashProcedure(199): jenkins-hbase4.apache.org,34773,1690229472389 had 0 regions 2023-07-24 20:11:23,041 INFO [zk-event-processor-pool-0] replication.ReplicationTrackerZKImpl$OtherRegionServerWatcher(124): /hbase/rs/jenkins-hbase4.apache.org,34773,1690229472389 znode expired, triggering replicatorRemoved event 2023-07-24 20:11:23,041 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,37489,1690229472216] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:11:23,043 INFO [PEWorker-3] procedure.ServerCrashProcedure(300): Splitting WALs pid=139, state=RUNNABLE:SERVER_CRASH_SPLIT_LOGS, locked=true; ServerCrashProcedure jenkins-hbase4.apache.org,34773,1690229472389, splitWal=true, meta=false, isMeta: false 2023-07-24 20:11:23,043 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:46693-0x10198f4f3ce001d, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46693,1690229472273 2023-07-24 20:11:23,043 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:38597-0x10198f4f3ce001e, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46693,1690229472273 2023-07-24 20:11:23,043 INFO [zk-event-processor-pool-0] replication.ReplicationTrackerZKImpl$OtherRegionServerWatcher(124): /hbase/rs/jenkins-hbase4.apache.org,34773,1690229472389 znode expired, triggering replicatorRemoved event 2023-07-24 20:11:23,043 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:41283-0x10198f4f3ce0028, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38597,1690229472335 2023-07-24 20:11:23,043 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:41283-0x10198f4f3ce0028, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41283,1690229481649 2023-07-24 20:11:23,043 INFO [zk-event-processor-pool-0] replication.ReplicationTrackerZKImpl$OtherRegionServerWatcher(124): /hbase/rs/jenkins-hbase4.apache.org,34773,1690229472389 znode expired, triggering replicatorRemoved event 2023-07-24 20:11:23,043 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,37489,1690229472216] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/Group_testClearDeadServers_1855695895 2023-07-24 20:11:23,044 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:38597-0x10198f4f3ce001e, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38597,1690229472335 2023-07-24 20:11:23,044 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:41283-0x10198f4f3ce0028, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46693,1690229472273 2023-07-24 20:11:23,044 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,37489,1690229472216] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:11:23,044 DEBUG [PEWorker-3] master.MasterWalManager(318): Renamed region directory: hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,34773,1690229472389-splitting 2023-07-24 20:11:23,044 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:38597-0x10198f4f3ce001e, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41283,1690229481649 2023-07-24 20:11:23,045 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:46693-0x10198f4f3ce001d, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38597,1690229472335 2023-07-24 20:11:23,045 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,37489,1690229472216] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 6 2023-07-24 20:11:23,045 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:46693-0x10198f4f3ce001d, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41283,1690229481649 2023-07-24 20:11:23,045 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:38597-0x10198f4f3ce001e, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46693,1690229472273 2023-07-24 20:11:23,046 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:46693-0x10198f4f3ce001d, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46693,1690229472273 2023-07-24 20:11:23,046 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,34773,1690229472389-splitting dir is empty, no logs to split. 2023-07-24 20:11:23,046 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,34773,1690229472389 WAL count=0, meta=false 2023-07-24 20:11:23,046 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-24 20:11:23,048 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,37489,1690229472216] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(792): Updated with servers: 1 2023-07-24 20:11:23,051 INFO [PEWorker-3] master.SplitLogManager(171): hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,34773,1690229472389-splitting dir is empty, no logs to split. 2023-07-24 20:11:23,051 INFO [PEWorker-3] master.SplitWALManager(106): jenkins-hbase4.apache.org,34773,1690229472389 WAL count=0, meta=false 2023-07-24 20:11:23,051 DEBUG [PEWorker-3] procedure.ServerCrashProcedure(290): Check if jenkins-hbase4.apache.org,34773,1690229472389 WAL splitting is done? wals=0, meta=false 2023-07-24 20:11:23,054 INFO [PEWorker-3] procedure.ServerCrashProcedure(282): Remove WAL directory for jenkins-hbase4.apache.org,34773,1690229472389 failed, ignore...File hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,34773,1690229472389-splitting does not exist. 2023-07-24 20:11:23,055 INFO [PEWorker-3] procedure.ServerCrashProcedure(251): removed crashed server jenkins-hbase4.apache.org,34773,1690229472389 after splitting done 2023-07-24 20:11:23,055 DEBUG [PEWorker-3] master.DeadServer(114): Removed jenkins-hbase4.apache.org,34773,1690229472389 from processing; numProcessing=0 2023-07-24 20:11:23,057 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=139, state=SUCCESS; ServerCrashProcedure jenkins-hbase4.apache.org,34773,1690229472389, splitWal=true, meta=false in 20 msec 2023-07-24 20:11:23,113 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(2362): Client=jenkins//172.31.14.131 clear dead region servers. 2023-07-24 20:11:23,119 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:11:23,119 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/Group_testClearDeadServers_1855695895 2023-07-24 20:11:23,120 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:11:23,120 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 6 2023-07-24 20:11:23,124 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminServer(609): Remove decommissioned servers [jenkins-hbase4.apache.org:34773] from RSGroup done 2023-07-24 20:11:23,126 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=Group_testClearDeadServers_1855695895 2023-07-24 20:11:23,126 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-24 20:11:23,128 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38597] ipc.CallRunner(144): callId: 72 service: ClientService methodName: Scan size: 146 connection: 172.31.14.131:43856 deadline: 1690229543128, exception=org.apache.hadoop.hbase.exceptions.RegionMovedException: Region moved to: hostname=jenkins-hbase4.apache.org port=46693 startCode=1690229472273. As of locationSeqNum=18. 2023-07-24 20:11:23,217 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34773-0x10198f4f3ce001f, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 20:11:23,217 INFO [RS:2;jenkins-hbase4:34773] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,34773,1690229472389; zookeeper connection closed. 2023-07-24 20:11:23,217 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:34773-0x10198f4f3ce001f, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 20:11:23,217 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@3a905f92] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@3a905f92 2023-07-24 20:11:23,256 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:11:23,256 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:11:23,257 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-24 20:11:23,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-24 20:11:23,257 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-24 20:11:23,259 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:41283, jenkins-hbase4.apache.org:38597] to rsgroup default 2023-07-24 20:11:23,264 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:11:23,264 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/Group_testClearDeadServers_1855695895 2023-07-24 20:11:23,265 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:11:23,265 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 6 2023-07-24 20:11:23,267 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminServer(286): Moving 0 region(s) to group Group_testClearDeadServers_1855695895, current retry=0 2023-07-24 20:11:23,267 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminServer(261): All regions from [jenkins-hbase4.apache.org,38597,1690229472335, jenkins-hbase4.apache.org,41283,1690229481649] are moved back to Group_testClearDeadServers_1855695895 2023-07-24 20:11:23,267 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminServer(438): Move servers done: Group_testClearDeadServers_1855695895 => default 2023-07-24 20:11:23,267 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-24 20:11:23,268 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup Group_testClearDeadServers_1855695895 2023-07-24 20:11:23,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:11:23,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:11:23,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 5 2023-07-24 20:11:23,282 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-24 20:11:23,283 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(231): Client=jenkins//172.31.14.131 move tables [] to rsgroup default 2023-07-24 20:11:23,283 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminServer(448): moveTables() passed an empty set. Ignoring. 2023-07-24 20:11:23,283 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveTables 2023-07-24 20:11:23,285 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [] to rsgroup default 2023-07-24 20:11:23,285 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.MoveServers 2023-07-24 20:11:23,287 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(272): Client=jenkins//172.31.14.131 remove rsgroup master 2023-07-24 20:11:23,296 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:11:23,296 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 3 2023-07-24 20:11:23,298 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.RemoveRSGroup 2023-07-24 20:11:23,302 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase(152): Restoring servers: 1 2023-07-24 20:11:23,318 INFO [Listener at localhost/44145] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-07-24 20:11:23,319 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 20:11:23,319 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-24 20:11:23,319 INFO [Listener at localhost/44145] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-24 20:11:23,319 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 20:11:23,319 INFO [Listener at localhost/44145] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-24 20:11:23,319 INFO [Listener at localhost/44145] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-24 20:11:23,323 INFO [Listener at localhost/44145] ipc.NettyRpcServer(120): Bind to /172.31.14.131:33773 2023-07-24 20:11:23,323 INFO [Listener at localhost/44145] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-24 20:11:23,346 DEBUG [Listener at localhost/44145] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-24 20:11:23,347 INFO [Listener at localhost/44145] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:11:23,349 INFO [Listener at localhost/44145] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 20:11:23,350 INFO [Listener at localhost/44145] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:33773 connecting to ZooKeeper ensemble=127.0.0.1:62881 2023-07-24 20:11:23,361 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:337730x0, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-24 20:11:23,363 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(162): regionserver:337730x0, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-24 20:11:23,365 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:33773-0x10198f4f3ce002a connected 2023-07-24 20:11:23,365 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(162): regionserver:33773-0x10198f4f3ce002a, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/running 2023-07-24 20:11:23,367 DEBUG [Listener at localhost/44145] zookeeper.ZKUtil(164): regionserver:33773-0x10198f4f3ce002a, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-24 20:11:23,374 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=33773 2023-07-24 20:11:23,374 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=33773 2023-07-24 20:11:23,375 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=33773 2023-07-24 20:11:23,378 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=33773 2023-07-24 20:11:23,378 DEBUG [Listener at localhost/44145] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=33773 2023-07-24 20:11:23,381 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter) 2023-07-24 20:11:23,381 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'clickjackingprevention' (class=org.apache.hadoop.hbase.http.ClickjackingPreventionFilter) 2023-07-24 20:11:23,381 INFO [Listener at localhost/44145] http.HttpServer(900): Added global filter 'securityheaders' (class=org.apache.hadoop.hbase.http.SecurityHeadersFilter) 2023-07-24 20:11:23,381 INFO [Listener at localhost/44145] http.HttpServer(879): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context regionserver 2023-07-24 20:11:23,382 INFO [Listener at localhost/44145] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs 2023-07-24 20:11:23,382 INFO [Listener at localhost/44145] http.HttpServer(886): Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static 2023-07-24 20:11:23,382 INFO [Listener at localhost/44145] http.HttpServer(783): ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint. 2023-07-24 20:11:23,383 INFO [Listener at localhost/44145] http.HttpServer(1146): Jetty bound to port 43845 2023-07-24 20:11:23,383 INFO [Listener at localhost/44145] server.Server(375): jetty-9.4.50.v20221201; built: 2022-12-01T22:07:03.915Z; git: da9a0b30691a45daf90a9f17b5defa2f1434f882; jvm 1.8.0_362-b09 2023-07-24 20:11:23,391 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:11:23,391 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@674b8a77{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/hadoop.log.dir/,AVAILABLE} 2023-07-24 20:11:23,392 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:11:23,392 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.s.ServletContextHandler@3f6c63a2{static,/static,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/static/,AVAILABLE} 2023-07-24 20:11:23,401 INFO [Listener at localhost/44145] webapp.StandardDescriptorProcessor(277): NO JSP Support for /, did not find org.apache.hbase.thirdparty.org.eclipse.jetty.jsp.JettyJspServlet 2023-07-24 20:11:23,403 INFO [Listener at localhost/44145] session.DefaultSessionIdManager(334): DefaultSessionIdManager workerName=node0 2023-07-24 20:11:23,403 INFO [Listener at localhost/44145] session.DefaultSessionIdManager(339): No SessionScavenger set, using defaults 2023-07-24 20:11:23,403 INFO [Listener at localhost/44145] session.HouseKeeper(132): node0 Scavenging every 600000ms 2023-07-24 20:11:23,405 INFO [Listener at localhost/44145] http.SecurityHeadersFilter(48): Added security headers filter 2023-07-24 20:11:23,406 INFO [Listener at localhost/44145] handler.ContextHandler(921): Started o.a.h.t.o.e.j.w.WebAppContext@6a653179{regionserver,/,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver/,AVAILABLE}{file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver} 2023-07-24 20:11:23,408 INFO [Listener at localhost/44145] server.AbstractConnector(333): Started ServerConnector@1ba711bb{HTTP/1.1, (http/1.1)}{0.0.0.0:43845} 2023-07-24 20:11:23,408 INFO [Listener at localhost/44145] server.Server(415): Started @51975ms 2023-07-24 20:11:23,413 INFO [RS:4;jenkins-hbase4:33773] regionserver.HRegionServer(951): ClusterId : dc54b028-3544-4133-8f0e-d550a19bc790 2023-07-24 20:11:23,413 DEBUG [RS:4;jenkins-hbase4:33773] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-24 20:11:23,415 DEBUG [RS:4;jenkins-hbase4:33773] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-24 20:11:23,415 DEBUG [RS:4;jenkins-hbase4:33773] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-24 20:11:23,417 DEBUG [RS:4;jenkins-hbase4:33773] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-24 20:11:23,418 DEBUG [RS:4;jenkins-hbase4:33773] zookeeper.ReadOnlyZKClient(139): Connect 0x6740111c to 127.0.0.1:62881 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-24 20:11:23,425 DEBUG [RS:4;jenkins-hbase4:33773] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6987e384, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=null 2023-07-24 20:11:23,425 DEBUG [RS:4;jenkins-hbase4:33773] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6db87a6c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-24 20:11:23,433 DEBUG [RS:4;jenkins-hbase4:33773] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:4;jenkins-hbase4:33773 2023-07-24 20:11:23,433 INFO [RS:4;jenkins-hbase4:33773] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-24 20:11:23,433 INFO [RS:4;jenkins-hbase4:33773] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-24 20:11:23,433 DEBUG [RS:4;jenkins-hbase4:33773] regionserver.HRegionServer(1022): About to register with Master. 2023-07-24 20:11:23,434 INFO [RS:4;jenkins-hbase4:33773] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase4.apache.org,37489,1690229472216 with isa=jenkins-hbase4.apache.org/172.31.14.131:33773, startcode=1690229483317 2023-07-24 20:11:23,434 DEBUG [RS:4;jenkins-hbase4:33773] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-24 20:11:23,436 INFO [RS-EventLoopGroup-13-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56521, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.12 (auth:SIMPLE), service=RegionServerStatusService 2023-07-24 20:11:23,437 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=37489] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,33773,1690229483317 2023-07-24 20:11:23,437 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,37489,1690229472216] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(787): Updating default servers. 2023-07-24 20:11:23,437 DEBUG [RS:4;jenkins-hbase4:33773] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4 2023-07-24 20:11:23,437 DEBUG [RS:4;jenkins-hbase4:33773] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:37609 2023-07-24 20:11:23,437 DEBUG [RS:4;jenkins-hbase4:33773] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=43499 2023-07-24 20:11:23,440 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:38597-0x10198f4f3ce001e, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:11:23,440 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:41283-0x10198f4f3ce0028, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:11:23,440 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:11:23,440 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:46693-0x10198f4f3ce001d, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:11:23,440 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,37489,1690229472216] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:11:23,440 DEBUG [RS:4;jenkins-hbase4:33773] zookeeper.ZKUtil(162): regionserver:33773-0x10198f4f3ce002a, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33773,1690229483317 2023-07-24 20:11:23,440 WARN [RS:4;jenkins-hbase4:33773] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-24 20:11:23,440 INFO [RS:4;jenkins-hbase4:33773] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-24 20:11:23,441 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,33773,1690229483317] 2023-07-24 20:11:23,441 DEBUG [RS:4;jenkins-hbase4:33773] regionserver.HRegionServer(1948): logDir=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/WALs/jenkins-hbase4.apache.org,33773,1690229483317 2023-07-24 20:11:23,441 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:41283-0x10198f4f3ce0028, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38597,1690229472335 2023-07-24 20:11:23,441 DEBUG [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,37489,1690229472216] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 2 2023-07-24 20:11:23,441 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:46693-0x10198f4f3ce001d, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38597,1690229472335 2023-07-24 20:11:23,441 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:38597-0x10198f4f3ce001e, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38597,1690229472335 2023-07-24 20:11:23,441 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:41283-0x10198f4f3ce0028, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33773,1690229483317 2023-07-24 20:11:23,443 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:46693-0x10198f4f3ce001d, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33773,1690229483317 2023-07-24 20:11:23,443 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:41283-0x10198f4f3ce0028, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41283,1690229481649 2023-07-24 20:11:23,443 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:41283-0x10198f4f3ce0028, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46693,1690229472273 2023-07-24 20:11:23,443 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:38597-0x10198f4f3ce001e, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33773,1690229483317 2023-07-24 20:11:23,443 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:46693-0x10198f4f3ce001d, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41283,1690229481649 2023-07-24 20:11:23,445 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:38597-0x10198f4f3ce001e, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41283,1690229481649 2023-07-24 20:11:23,445 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:38597-0x10198f4f3ce001e, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46693,1690229472273 2023-07-24 20:11:23,443 INFO [org.apache.hadoop.hbase.rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread-jenkins-hbase4.apache.org,37489,1690229472216] rsgroup.RSGroupInfoManagerImpl$ServerEventsListenerThread(792): Updated with servers: 4 2023-07-24 20:11:23,447 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): regionserver:46693-0x10198f4f3ce001d, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46693,1690229472273 2023-07-24 20:11:23,451 DEBUG [RS:4;jenkins-hbase4:33773] zookeeper.ZKUtil(162): regionserver:33773-0x10198f4f3ce002a, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,38597,1690229472335 2023-07-24 20:11:23,452 DEBUG [RS:4;jenkins-hbase4:33773] zookeeper.ZKUtil(162): regionserver:33773-0x10198f4f3ce002a, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,33773,1690229483317 2023-07-24 20:11:23,452 DEBUG [RS:4;jenkins-hbase4:33773] zookeeper.ZKUtil(162): regionserver:33773-0x10198f4f3ce002a, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,41283,1690229481649 2023-07-24 20:11:23,452 DEBUG [RS:4;jenkins-hbase4:33773] zookeeper.ZKUtil(162): regionserver:33773-0x10198f4f3ce002a, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,46693,1690229472273 2023-07-24 20:11:23,453 DEBUG [RS:4;jenkins-hbase4:33773] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-24 20:11:23,453 INFO [RS:4;jenkins-hbase4:33773] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-24 20:11:23,459 INFO [RS:4;jenkins-hbase4:33773] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-24 20:11:23,459 INFO [RS:4;jenkins-hbase4:33773] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-24 20:11:23,459 INFO [RS:4;jenkins-hbase4:33773] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:23,459 INFO [RS:4;jenkins-hbase4:33773] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-24 20:11:23,461 INFO [RS:4;jenkins-hbase4:33773] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:23,468 DEBUG [RS:4;jenkins-hbase4:33773] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:23,468 DEBUG [RS:4;jenkins-hbase4:33773] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:23,468 DEBUG [RS:4;jenkins-hbase4:33773] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:23,468 DEBUG [RS:4;jenkins-hbase4:33773] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:23,468 DEBUG [RS:4;jenkins-hbase4:33773] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:23,468 DEBUG [RS:4;jenkins-hbase4:33773] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-07-24 20:11:23,468 DEBUG [RS:4;jenkins-hbase4:33773] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:23,468 DEBUG [RS:4;jenkins-hbase4:33773] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:23,468 DEBUG [RS:4;jenkins-hbase4:33773] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:23,468 DEBUG [RS:4;jenkins-hbase4:33773] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-07-24 20:11:23,470 INFO [RS:4;jenkins-hbase4:33773] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:23,471 INFO [RS:4;jenkins-hbase4:33773] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:23,471 INFO [RS:4;jenkins-hbase4:33773] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:23,483 INFO [RS:4;jenkins-hbase4:33773] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-24 20:11:23,483 INFO [RS:4;jenkins-hbase4:33773] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,33773,1690229483317-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 20:11:23,493 INFO [RS:4;jenkins-hbase4:33773] regionserver.Replication(203): jenkins-hbase4.apache.org,33773,1690229483317 started 2023-07-24 20:11:23,493 INFO [RS:4;jenkins-hbase4:33773] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,33773,1690229483317, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:33773, sessionid=0x10198f4f3ce002a 2023-07-24 20:11:23,493 DEBUG [RS:4;jenkins-hbase4:33773] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-24 20:11:23,493 DEBUG [RS:4;jenkins-hbase4:33773] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,33773,1690229483317 2023-07-24 20:11:23,493 DEBUG [RS:4;jenkins-hbase4:33773] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33773,1690229483317' 2023-07-24 20:11:23,493 DEBUG [RS:4;jenkins-hbase4:33773] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 20:11:23,494 DEBUG [RS:4;jenkins-hbase4:33773] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 20:11:23,494 DEBUG [RS:4;jenkins-hbase4:33773] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-24 20:11:23,494 DEBUG [RS:4;jenkins-hbase4:33773] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-24 20:11:23,494 DEBUG [RS:4;jenkins-hbase4:33773] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,33773,1690229483317 2023-07-24 20:11:23,494 DEBUG [RS:4;jenkins-hbase4:33773] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,33773,1690229483317' 2023-07-24 20:11:23,494 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(252): Client=jenkins//172.31.14.131 add rsgroup master 2023-07-24 20:11:23,494 DEBUG [RS:4;jenkins-hbase4:33773] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-24 20:11:23,495 DEBUG [RS:4;jenkins-hbase4:33773] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-24 20:11:23,495 DEBUG [RS:4;jenkins-hbase4:33773] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-24 20:11:23,495 INFO [RS:4;jenkins-hbase4:33773] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-24 20:11:23,495 INFO [RS:4;jenkins-hbase4:33773] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-24 20:11:23,496 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/default 2023-07-24 20:11:23,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupInfoManagerImpl(662): Updating znode: /hbase/rsgroup/master 2023-07-24 20:11:23,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupInfoManagerImpl(668): Writing ZK GroupInfo count: 4 2023-07-24 20:11:23,499 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.AddRSGroup 2023-07-24 20:11:23,502 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:11:23,502 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:11:23,504 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(206): Client=jenkins//172.31.14.131 move servers [jenkins-hbase4.apache.org:37489] to rsgroup master 2023-07-24 20:11:23,504 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:37489 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-24 20:11:23,504 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] ipc.CallRunner(144): callId: 104 service: MasterService methodName: ExecMasterService size: 118 connection: 172.31.14.131:58688 deadline: 1690230683504, exception=org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:37489 is either offline or it does not exist. 2023-07-24 20:11:23,505 WARN [Listener at localhost/44145] rsgroup.TestRSGroupsBase(163): Got this on setup, FYI org.apache.hadoop.hbase.constraint.ConstraintException: org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:37489 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at sun.reflect.GeneratedConstructorAccessor62.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.instantiateException(RemoteWithExtrasException.java:97) at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:87) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:376) at org.apache.hadoop.hbase.shaded.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:364) at org.apache.hadoop.hbase.client.MasterCallable.call(MasterCallable.java:101) at org.apache.hadoop.hbase.client.HBaseAdmin$74.callExecService(HBaseAdmin.java:2985) at org.apache.hadoop.hbase.client.SyncCoprocessorRpcChannel.callBlockingMethod(SyncCoprocessorRpcChannel.java:62) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService$BlockingStub.moveServers(RSGroupAdminProtos.java:16658) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminClient.moveServers(RSGroupAdminClient.java:108) at org.apache.hadoop.hbase.rsgroup.VerifyingRSGroupAdminClient.moveServers(VerifyingRSGroupAdminClient.java:77) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase.tearDownAfterMethod(TestRSGroupsBase.java:161) at org.apache.hadoop.hbase.rsgroup.TestRSGroupsBasics.afterMethod(TestRSGroupsBasics.java:82) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56) at org.junit.internal.runners.statements.RunAfters.invokeMethod(RunAfters.java:46) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61) at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306) at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63) at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329) at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:750) Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.constraint.ConstraintException): org.apache.hadoop.hbase.constraint.ConstraintException: Server jenkins-hbase4.apache.org:37489 is either offline or it does not exist. at org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveServers(RSGroupAdminServer.java:408) at org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveServers(RSGroupAdminEndpoint.java:213) at org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:16193) at org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:900) at org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.onCallFinished(AbstractRpcClient.java:382) at org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$100(AbstractRpcClient.java:88) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:416) at org.apache.hadoop.hbase.ipc.AbstractRpcClient$3.run(AbstractRpcClient.java:412) at org.apache.hadoop.hbase.ipc.Call.callComplete(Call.java:115) at org.apache.hadoop.hbase.ipc.Call.setException(Call.java:130) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.readResponse(NettyRpcDuplexHandler.java:162) at org.apache.hadoop.hbase.ipc.NettyRpcDuplexHandler.channelRead(NettyRpcDuplexHandler.java:192) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346) at org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) at org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650) at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ... 1 more 2023-07-24 20:11:23,509 INFO [Listener at localhost/44145] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 20:11:23,510 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(316): Client=jenkins//172.31.14.131 list rsgroup 2023-07-24 20:11:23,510 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.ListRSGroupInfos 2023-07-24 20:11:23,511 INFO [Listener at localhost/44145] rsgroup.TestRSGroupsBase$2(169): Waiting for cleanup to finish [Name:default, Servers:[jenkins-hbase4.apache.org:33773, jenkins-hbase4.apache.org:38597, jenkins-hbase4.apache.org:41283, jenkins-hbase4.apache.org:46693], Tables:[hbase:meta, hbase:quota, hbase:namespace, hbase:rsgroup], Configurations:{}, Name:master, Servers:[], Tables:[], Configurations:{}] 2023-07-24 20:11:23,511 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl(165): Client=jenkins//172.31.14.131 initiates rsgroup info retrieval, group=default 2023-07-24 20:11:23,511 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37489] master.MasterRpcServices(912): User jenkins (auth:SIMPLE) (remote address: /172.31.14.131) master service request for RSGroupAdminService.GetRSGroupInfo 2023-07-24 20:11:23,532 INFO [Listener at localhost/44145] hbase.ResourceChecker(175): after: rsgroup.TestRSGroupsBasics#testClearDeadServers Thread=568 (was 557) - Thread LEAK? -, OpenFileDescriptor=854 (was 872), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=370 (was 370), ProcessCount=174 (was 174), AvailableMemoryMB=7684 (was 7762) 2023-07-24 20:11:23,532 WARN [Listener at localhost/44145] hbase.ResourceChecker(130): Thread=568 is superior to 500 2023-07-24 20:11:23,532 INFO [Listener at localhost/44145] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-07-24 20:11:23,532 INFO [Listener at localhost/44145] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-07-24 20:11:23,532 DEBUG [Listener at localhost/44145] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5e38523b to 127.0.0.1:62881 2023-07-24 20:11:23,532 DEBUG [Listener at localhost/44145] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:11:23,532 DEBUG [Listener at localhost/44145] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-24 20:11:23,532 DEBUG [Listener at localhost/44145] util.JVMClusterUtil(257): Found active master hash=575093195, stopped=false 2023-07-24 20:11:23,532 DEBUG [Listener at localhost/44145] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint 2023-07-24 20:11:23,533 DEBUG [Listener at localhost/44145] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.rsgroup.TestRSGroupsBase$CPMasterObserver 2023-07-24 20:11:23,533 INFO [Listener at localhost/44145] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,37489,1690229472216 2023-07-24 20:11:23,535 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:33773-0x10198f4f3ce002a, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-24 20:11:23,535 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:46693-0x10198f4f3ce001d, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-24 20:11:23,535 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-24 20:11:23,535 INFO [Listener at localhost/44145] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 20:11:23,535 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:41283-0x10198f4f3ce0028, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-24 20:11:23,535 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:38597-0x10198f4f3ce001e, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-24 20:11:23,536 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 20:11:23,537 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 20:11:23,537 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:33773-0x10198f4f3ce002a, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 20:11:23,538 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:46693-0x10198f4f3ce001d, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 20:11:23,538 DEBUG [Listener at localhost/44145] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0bb4fccf to 127.0.0.1:62881 2023-07-24 20:11:23,538 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41283-0x10198f4f3ce0028, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 20:11:23,538 DEBUG [Listener at localhost/44145] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:11:23,538 INFO [Listener at localhost/44145] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,46693,1690229472273' ***** 2023-07-24 20:11:23,538 INFO [Listener at localhost/44145] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-24 20:11:23,538 INFO [Listener at localhost/44145] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,38597,1690229472335' ***** 2023-07-24 20:11:23,538 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:38597-0x10198f4f3ce001e, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 20:11:23,538 INFO [RS:0;jenkins-hbase4:46693] regionserver.HRegionServer(1109): Stopping infoServer 2023-07-24 20:11:23,538 INFO [Listener at localhost/44145] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-24 20:11:23,539 INFO [Listener at localhost/44145] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,41283,1690229481649' ***** 2023-07-24 20:11:23,539 INFO [Listener at localhost/44145] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-24 20:11:23,540 INFO [Listener at localhost/44145] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,33773,1690229483317' ***** 2023-07-24 20:11:23,540 INFO [Listener at localhost/44145] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-24 20:11:23,540 INFO [RS:3;jenkins-hbase4:41283] regionserver.HRegionServer(1109): Stopping infoServer 2023-07-24 20:11:23,540 INFO [RS:1;jenkins-hbase4:38597] regionserver.HRegionServer(1109): Stopping infoServer 2023-07-24 20:11:23,540 INFO [RS:4;jenkins-hbase4:33773] regionserver.HRegionServer(1109): Stopping infoServer 2023-07-24 20:11:23,542 INFO [RS:0;jenkins-hbase4:46693] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.w.WebAppContext@5c3d90e7{regionserver,/,null,STOPPED}{file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver} 2023-07-24 20:11:23,544 INFO [RS:3;jenkins-hbase4:41283] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.w.WebAppContext@66477fc4{regionserver,/,null,STOPPED}{file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver} 2023-07-24 20:11:23,544 INFO [RS:3;jenkins-hbase4:41283] server.AbstractConnector(383): Stopped ServerConnector@76b99374{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-24 20:11:23,544 INFO [RS:3;jenkins-hbase4:41283] session.HouseKeeper(149): node0 Stopped scavenging 2023-07-24 20:11:23,547 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-24 20:11:23,548 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-24 20:11:23,548 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-24 20:11:23,549 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-24 20:11:23,553 INFO [RS:3;jenkins-hbase4:41283] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@2babb338{static,/static,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/static/,STOPPED} 2023-07-24 20:11:23,555 INFO [RS:0;jenkins-hbase4:46693] server.AbstractConnector(383): Stopped ServerConnector@237e24d4{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-24 20:11:23,556 INFO [RS:1;jenkins-hbase4:38597] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.w.WebAppContext@3e37ff10{regionserver,/,null,STOPPED}{file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver} 2023-07-24 20:11:23,556 INFO [RS:4;jenkins-hbase4:33773] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.w.WebAppContext@6a653179{regionserver,/,null,STOPPED}{file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/regionserver} 2023-07-24 20:11:23,556 INFO [RS:3;jenkins-hbase4:41283] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@4c9fd641{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/hadoop.log.dir/,STOPPED} 2023-07-24 20:11:23,556 INFO [RS:0;jenkins-hbase4:46693] session.HouseKeeper(149): node0 Stopped scavenging 2023-07-24 20:11:23,556 INFO [RS:4;jenkins-hbase4:33773] server.AbstractConnector(383): Stopped ServerConnector@1ba711bb{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-24 20:11:23,556 INFO [RS:4;jenkins-hbase4:33773] session.HouseKeeper(149): node0 Stopped scavenging 2023-07-24 20:11:23,557 INFO [RS:0;jenkins-hbase4:46693] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@dfee794{static,/static,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/static/,STOPPED} 2023-07-24 20:11:23,558 INFO [RS:4;jenkins-hbase4:33773] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@3f6c63a2{static,/static,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/static/,STOPPED} 2023-07-24 20:11:23,556 INFO [RS:1;jenkins-hbase4:38597] server.AbstractConnector(383): Stopped ServerConnector@34284301{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-24 20:11:23,559 INFO [RS:4;jenkins-hbase4:33773] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@674b8a77{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/hadoop.log.dir/,STOPPED} 2023-07-24 20:11:23,559 INFO [RS:0;jenkins-hbase4:46693] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@750dea8b{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/hadoop.log.dir/,STOPPED} 2023-07-24 20:11:23,558 INFO [RS:3;jenkins-hbase4:41283] regionserver.HeapMemoryManager(220): Stopping 2023-07-24 20:11:23,559 INFO [RS:1;jenkins-hbase4:38597] session.HouseKeeper(149): node0 Stopped scavenging 2023-07-24 20:11:23,560 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-24 20:11:23,560 INFO [RS:3;jenkins-hbase4:41283] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-24 20:11:23,560 INFO [RS:4;jenkins-hbase4:33773] regionserver.HeapMemoryManager(220): Stopping 2023-07-24 20:11:23,560 INFO [RS:0;jenkins-hbase4:46693] regionserver.HeapMemoryManager(220): Stopping 2023-07-24 20:11:23,560 INFO [RS:3;jenkins-hbase4:41283] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-24 20:11:23,561 INFO [RS:1;jenkins-hbase4:38597] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@3f748609{static,/static,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/static/,STOPPED} 2023-07-24 20:11:23,561 INFO [RS:3;jenkins-hbase4:41283] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,41283,1690229481649 2023-07-24 20:11:23,561 DEBUG [RS:3;jenkins-hbase4:41283] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0621c7c0 to 127.0.0.1:62881 2023-07-24 20:11:23,562 INFO [RS:1;jenkins-hbase4:38597] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@3006b674{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/hadoop.log.dir/,STOPPED} 2023-07-24 20:11:23,562 INFO [RS:0;jenkins-hbase4:46693] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-24 20:11:23,562 INFO [RS:0;jenkins-hbase4:46693] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-24 20:11:23,563 INFO [RS:0;jenkins-hbase4:46693] regionserver.HRegionServer(3305): Received CLOSE for 4fa2a21e5a49480a149a055b0fc39515 2023-07-24 20:11:23,562 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-24 20:11:23,562 INFO [RS:4;jenkins-hbase4:33773] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-24 20:11:23,562 DEBUG [RS:3;jenkins-hbase4:41283] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:11:23,563 INFO [RS:4;jenkins-hbase4:33773] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-24 20:11:23,563 INFO [RS:3;jenkins-hbase4:41283] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,41283,1690229481649; all regions closed. 2023-07-24 20:11:23,563 INFO [RS:4;jenkins-hbase4:33773] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,33773,1690229483317 2023-07-24 20:11:23,563 DEBUG [RS:4;jenkins-hbase4:33773] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x6740111c to 127.0.0.1:62881 2023-07-24 20:11:23,563 DEBUG [RS:4;jenkins-hbase4:33773] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:11:23,563 INFO [RS:0;jenkins-hbase4:46693] regionserver.HRegionServer(3305): Received CLOSE for 324410cf2a29c2dca1a19e8570799d47 2023-07-24 20:11:23,563 INFO [RS:4;jenkins-hbase4:33773] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,33773,1690229483317; all regions closed. 2023-07-24 20:11:23,563 INFO [RS:0;jenkins-hbase4:46693] regionserver.HRegionServer(3305): Received CLOSE for ec4f892f95beff2c1aa7863da43a18fa 2023-07-24 20:11:23,563 INFO [RS:0;jenkins-hbase4:46693] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,46693,1690229472273 2023-07-24 20:11:23,563 DEBUG [RS:0;jenkins-hbase4:46693] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1c312feb to 127.0.0.1:62881 2023-07-24 20:11:23,563 DEBUG [RS:0;jenkins-hbase4:46693] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:11:23,563 INFO [RS:0;jenkins-hbase4:46693] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-24 20:11:23,563 INFO [RS:0;jenkins-hbase4:46693] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-24 20:11:23,564 INFO [RS:0;jenkins-hbase4:46693] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-24 20:11:23,564 INFO [RS:0;jenkins-hbase4:46693] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-24 20:11:23,564 DEBUG [RS:4;jenkins-hbase4:33773] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:11:23,564 INFO [RS:4;jenkins-hbase4:33773] regionserver.LeaseManager(133): Closed leases 2023-07-24 20:11:23,564 INFO [RS:1;jenkins-hbase4:38597] regionserver.HeapMemoryManager(220): Stopping 2023-07-24 20:11:23,564 INFO [RS:0;jenkins-hbase4:46693] regionserver.HRegionServer(1474): Waiting on 4 regions to close 2023-07-24 20:11:23,564 INFO [RS:4;jenkins-hbase4:33773] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-24 20:11:23,565 DEBUG [RS:0;jenkins-hbase4:46693] regionserver.HRegionServer(1478): Online Regions={4fa2a21e5a49480a149a055b0fc39515=hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515., 1588230740=hbase:meta,,1.1588230740, 324410cf2a29c2dca1a19e8570799d47=hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47., ec4f892f95beff2c1aa7863da43a18fa=hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa.} 2023-07-24 20:11:23,565 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 4fa2a21e5a49480a149a055b0fc39515, disabling compactions & flushes 2023-07-24 20:11:23,565 DEBUG [RS:0;jenkins-hbase4:46693] regionserver.HRegionServer(1504): Waiting on 1588230740, 324410cf2a29c2dca1a19e8570799d47, 4fa2a21e5a49480a149a055b0fc39515, ec4f892f95beff2c1aa7863da43a18fa 2023-07-24 20:11:23,565 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515. 2023-07-24 20:11:23,566 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-24 20:11:23,566 INFO [RS:4;jenkins-hbase4:33773] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-24 20:11:23,566 INFO [RS:4;jenkins-hbase4:33773] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-24 20:11:23,566 INFO [RS:4;jenkins-hbase4:33773] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-24 20:11:23,565 INFO [RS:1;jenkins-hbase4:38597] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-24 20:11:23,568 INFO [RS:1;jenkins-hbase4:38597] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-24 20:11:23,568 INFO [RS:1;jenkins-hbase4:38597] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,38597,1690229472335 2023-07-24 20:11:23,568 DEBUG [RS:1;jenkins-hbase4:38597] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x47c01838 to 127.0.0.1:62881 2023-07-24 20:11:23,568 DEBUG [RS:1;jenkins-hbase4:38597] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:11:23,568 INFO [RS:1;jenkins-hbase4:38597] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,38597,1690229472335; all regions closed. 2023-07-24 20:11:23,566 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-24 20:11:23,569 INFO [RS:4;jenkins-hbase4:33773] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:33773 2023-07-24 20:11:23,566 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-24 20:11:23,569 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-24 20:11:23,566 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515. 2023-07-24 20:11:23,569 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-24 20:11:23,569 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-24 20:11:23,569 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=5.75 KB heapSize=10.19 KB 2023-07-24 20:11:23,569 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515. after waiting 0 ms 2023-07-24 20:11:23,569 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515. 2023-07-24 20:11:23,572 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-24 20:11:23,577 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:38597-0x10198f4f3ce001e, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,33773,1690229483317 2023-07-24 20:11:23,577 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:38597-0x10198f4f3ce001e, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:11:23,577 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:33773-0x10198f4f3ce002a, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,33773,1690229483317 2023-07-24 20:11:23,577 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:11:23,577 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:41283-0x10198f4f3ce0028, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,33773,1690229483317 2023-07-24 20:11:23,577 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:41283-0x10198f4f3ce0028, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:11:23,577 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:33773-0x10198f4f3ce002a, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:11:23,577 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:46693-0x10198f4f3ce001d, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,33773,1690229483317 2023-07-24 20:11:23,577 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:46693-0x10198f4f3ce001d, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:11:23,578 DEBUG [RS:3;jenkins-hbase4:41283] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/oldWALs 2023-07-24 20:11:23,578 INFO [RS:3;jenkins-hbase4:41283] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C41283%2C1690229481649:(num 1690229481926) 2023-07-24 20:11:23,578 DEBUG [RS:3;jenkins-hbase4:41283] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:11:23,578 INFO [RS:3;jenkins-hbase4:41283] regionserver.LeaseManager(133): Closed leases 2023-07-24 20:11:23,578 INFO [RS:3;jenkins-hbase4:41283] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-24 20:11:23,578 INFO [RS:3;jenkins-hbase4:41283] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-24 20:11:23,578 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-24 20:11:23,578 INFO [RS:3;jenkins-hbase4:41283] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-24 20:11:23,578 INFO [RS:3;jenkins-hbase4:41283] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-24 20:11:23,578 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,33773,1690229483317] 2023-07-24 20:11:23,579 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,33773,1690229483317; numProcessing=1 2023-07-24 20:11:23,581 INFO [RS:3;jenkins-hbase4:41283] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:41283 2023-07-24 20:11:23,582 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-24 20:11:23,582 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,33773,1690229483317 already deleted, retry=false 2023-07-24 20:11:23,582 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,33773,1690229483317 expired; onlineServers=3 2023-07-24 20:11:23,584 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:38597-0x10198f4f3ce001e, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41283,1690229481649 2023-07-24 20:11:23,584 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:41283-0x10198f4f3ce0028, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41283,1690229481649 2023-07-24 20:11:23,584 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:46693-0x10198f4f3ce001d, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,41283,1690229481649 2023-07-24 20:11:23,585 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:11:23,586 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,41283,1690229481649] 2023-07-24 20:11:23,586 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,41283,1690229481649; numProcessing=2 2023-07-24 20:11:23,588 DEBUG [RS:1;jenkins-hbase4:38597] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/oldWALs 2023-07-24 20:11:23,588 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/quota/4fa2a21e5a49480a149a055b0fc39515/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=7 2023-07-24 20:11:23,588 INFO [RS:1;jenkins-hbase4:38597] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C38597%2C1690229472335:(num 1690229472968) 2023-07-24 20:11:23,588 DEBUG [RS:1;jenkins-hbase4:38597] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:11:23,589 INFO [RS:1;jenkins-hbase4:38597] regionserver.LeaseManager(133): Closed leases 2023-07-24 20:11:23,589 INFO [RS:1;jenkins-hbase4:38597] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-24 20:11:23,589 INFO [RS:1;jenkins-hbase4:38597] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-24 20:11:23,589 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-24 20:11:23,589 INFO [RS:1;jenkins-hbase4:38597] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-24 20:11:23,589 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515. 2023-07-24 20:11:23,589 INFO [RS:1;jenkins-hbase4:38597] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-24 20:11:23,589 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 4fa2a21e5a49480a149a055b0fc39515: 2023-07-24 20:11:23,590 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:quota,,1690229465966.4fa2a21e5a49480a149a055b0fc39515. 2023-07-24 20:11:23,590 INFO [RS:1;jenkins-hbase4:38597] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:38597 2023-07-24 20:11:23,591 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 324410cf2a29c2dca1a19e8570799d47, disabling compactions & flushes 2023-07-24 20:11:23,591 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. 2023-07-24 20:11:23,591 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,41283,1690229481649 already deleted, retry=false 2023-07-24 20:11:23,591 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. 2023-07-24 20:11:23,591 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,41283,1690229481649 expired; onlineServers=2 2023-07-24 20:11:23,591 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. after waiting 0 ms 2023-07-24 20:11:23,591 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. 2023-07-24 20:11:23,592 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:46693-0x10198f4f3ce001d, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,38597,1690229472335 2023-07-24 20:11:23,592 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:38597-0x10198f4f3ce001e, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,38597,1690229472335 2023-07-24 20:11:23,592 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:11:23,594 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,38597,1690229472335] 2023-07-24 20:11:23,594 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,38597,1690229472335; numProcessing=3 2023-07-24 20:11:23,597 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,38597,1690229472335 already deleted, retry=false 2023-07-24 20:11:23,597 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,38597,1690229472335 expired; onlineServers=1 2023-07-24 20:11:23,600 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.75 KB at sequenceid=177 (bloomFilter=false), to=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/.tmp/info/61cfaf006d3d4779bbed2be0935d1ada 2023-07-24 20:11:23,601 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/namespace/324410cf2a29c2dca1a19e8570799d47/recovered.edits/23.seqid, newMaxSeqId=23, maxSeqId=20 2023-07-24 20:11:23,601 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. 2023-07-24 20:11:23,601 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 324410cf2a29c2dca1a19e8570799d47: 2023-07-24 20:11:23,602 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1690229442518.324410cf2a29c2dca1a19e8570799d47. 2023-07-24 20:11:23,602 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ec4f892f95beff2c1aa7863da43a18fa, disabling compactions & flushes 2023-07-24 20:11:23,602 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. 2023-07-24 20:11:23,602 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. 2023-07-24 20:11:23,602 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. after waiting 0 ms 2023-07-24 20:11:23,602 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. 2023-07-24 20:11:23,602 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing ec4f892f95beff2c1aa7863da43a18fa 1/1 column families, dataSize=4.27 KB heapSize=7.02 KB 2023-07-24 20:11:23,607 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/.tmp/info/61cfaf006d3d4779bbed2be0935d1ada as hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/info/61cfaf006d3d4779bbed2be0935d1ada 2023-07-24 20:11:23,613 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/info/61cfaf006d3d4779bbed2be0935d1ada, entries=38, sequenceid=177, filesize=9.1 K 2023-07-24 20:11:23,614 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~5.75 KB/5886, heapSize ~9.67 KB/9904, currentSize=0 B/0 for 1588230740 in 44ms, sequenceid=177, compaction requested=false 2023-07-24 20:11:23,624 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=4.27 KB at sequenceid=95 (bloomFilter=true), to=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/rsgroup/ec4f892f95beff2c1aa7863da43a18fa/.tmp/m/c3e68e2e4c684bb98dfaf47ca78e954e 2023-07-24 20:11:23,627 DEBUG [StoreCloser-hbase:meta,,1.1588230740-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/info/14fae0676b7b4b9dbef6a9c00b67e86e, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/info/71cc04e9686e4212a8a52f78335cc5c6, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/info/99bce8ea8bc2478680bf62bc041d6208] to archive 2023-07-24 20:11:23,627 DEBUG [StoreCloser-hbase:meta,,1.1588230740-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 20:11:23,630 DEBUG [StoreCloser-hbase:meta,,1.1588230740-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/info/14fae0676b7b4b9dbef6a9c00b67e86e to hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/archive/data/hbase/meta/1588230740/info/14fae0676b7b4b9dbef6a9c00b67e86e 2023-07-24 20:11:23,632 DEBUG [StoreCloser-hbase:meta,,1.1588230740-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/info/71cc04e9686e4212a8a52f78335cc5c6 to hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/archive/data/hbase/meta/1588230740/info/71cc04e9686e4212a8a52f78335cc5c6 2023-07-24 20:11:23,632 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c3e68e2e4c684bb98dfaf47ca78e954e 2023-07-24 20:11:23,633 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/rsgroup/ec4f892f95beff2c1aa7863da43a18fa/.tmp/m/c3e68e2e4c684bb98dfaf47ca78e954e as hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/rsgroup/ec4f892f95beff2c1aa7863da43a18fa/m/c3e68e2e4c684bb98dfaf47ca78e954e 2023-07-24 20:11:23,633 DEBUG [StoreCloser-hbase:meta,,1.1588230740-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/info/99bce8ea8bc2478680bf62bc041d6208 to hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/archive/data/hbase/meta/1588230740/info/99bce8ea8bc2478680bf62bc041d6208 2023-07-24 20:11:23,639 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c3e68e2e4c684bb98dfaf47ca78e954e 2023-07-24 20:11:23,639 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/rsgroup/ec4f892f95beff2c1aa7863da43a18fa/m/c3e68e2e4c684bb98dfaf47ca78e954e, entries=6, sequenceid=95, filesize=5.4 K 2023-07-24 20:11:23,639 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~4.27 KB/4376, heapSize ~7.01 KB/7176, currentSize=0 B/0 for ec4f892f95beff2c1aa7863da43a18fa in 37ms, sequenceid=95, compaction requested=true 2023-07-24 20:11:23,647 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/rsgroup/ec4f892f95beff2c1aa7863da43a18fa/recovered.edits/98.seqid, newMaxSeqId=98, maxSeqId=77 2023-07-24 20:11:23,648 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-24 20:11:23,648 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. 2023-07-24 20:11:23,648 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ec4f892f95beff2c1aa7863da43a18fa: 2023-07-24 20:11:23,649 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:rsgroup,,1690229442794.ec4f892f95beff2c1aa7863da43a18fa. 2023-07-24 20:11:23,677 DEBUG [StoreCloser-hbase:meta,,1.1588230740-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/table/a90c3b62969d4417a8b965913cf8fbbc, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/table/92a06725a50449fa925e17781e43f1e3, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/table/228b116fcd95447abde6f2baeea5c266] to archive 2023-07-24 20:11:23,678 DEBUG [StoreCloser-hbase:meta,,1.1588230740-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 20:11:23,680 DEBUG [StoreCloser-hbase:meta,,1.1588230740-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/table/a90c3b62969d4417a8b965913cf8fbbc to hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/archive/data/hbase/meta/1588230740/table/a90c3b62969d4417a8b965913cf8fbbc 2023-07-24 20:11:23,682 DEBUG [StoreCloser-hbase:meta,,1.1588230740-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/table/92a06725a50449fa925e17781e43f1e3 to hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/archive/data/hbase/meta/1588230740/table/92a06725a50449fa925e17781e43f1e3 2023-07-24 20:11:23,683 DEBUG [StoreCloser-hbase:meta,,1.1588230740-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/table/228b116fcd95447abde6f2baeea5c266 to hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/archive/data/hbase/meta/1588230740/table/228b116fcd95447abde6f2baeea5c266 2023-07-24 20:11:23,688 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/data/hbase/meta/1588230740/recovered.edits/180.seqid, newMaxSeqId=180, maxSeqId=158 2023-07-24 20:11:23,689 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-24 20:11:23,689 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-24 20:11:23,689 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-24 20:11:23,689 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-24 20:11:23,735 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:38597-0x10198f4f3ce001e, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 20:11:23,735 INFO [RS:1;jenkins-hbase4:38597] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,38597,1690229472335; zookeeper connection closed. 2023-07-24 20:11:23,735 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:38597-0x10198f4f3ce001e, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 20:11:23,735 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@87479ba] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@87479ba 2023-07-24 20:11:23,766 INFO [RS:0;jenkins-hbase4:46693] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,46693,1690229472273; all regions closed. 2023-07-24 20:11:23,775 DEBUG [RS:0;jenkins-hbase4:46693] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/oldWALs 2023-07-24 20:11:23,775 INFO [RS:0;jenkins-hbase4:46693] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C46693%2C1690229472273.meta:.meta(num 1690229473092) 2023-07-24 20:11:23,782 DEBUG [RS:0;jenkins-hbase4:46693] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/oldWALs 2023-07-24 20:11:23,783 INFO [RS:0;jenkins-hbase4:46693] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C46693%2C1690229472273:(num 1690229472977) 2023-07-24 20:11:23,783 DEBUG [RS:0;jenkins-hbase4:46693] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:11:23,783 INFO [RS:0;jenkins-hbase4:46693] regionserver.LeaseManager(133): Closed leases 2023-07-24 20:11:23,783 INFO [RS:0;jenkins-hbase4:46693] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-24 20:11:23,783 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-24 20:11:23,784 INFO [RS:0;jenkins-hbase4:46693] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:46693 2023-07-24 20:11:23,787 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:46693-0x10198f4f3ce001d, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,46693,1690229472273 2023-07-24 20:11:23,787 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 20:11:23,788 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,46693,1690229472273] 2023-07-24 20:11:23,788 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,46693,1690229472273; numProcessing=4 2023-07-24 20:11:23,789 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,46693,1690229472273 already deleted, retry=false 2023-07-24 20:11:23,789 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,46693,1690229472273 expired; onlineServers=0 2023-07-24 20:11:23,789 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase4.apache.org,37489,1690229472216' ***** 2023-07-24 20:11:23,790 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-24 20:11:23,790 DEBUG [M:0;jenkins-hbase4:37489] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2678a95b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=false, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-07-24 20:11:23,790 INFO [M:0;jenkins-hbase4:37489] regionserver.HRegionServer(1109): Stopping infoServer 2023-07-24 20:11:23,792 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-07-24 20:11:23,792 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 20:11:23,792 INFO [M:0;jenkins-hbase4:37489] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.w.WebAppContext@71da3018{master,/,null,STOPPED}{file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/master} 2023-07-24 20:11:23,792 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-24 20:11:23,792 INFO [M:0;jenkins-hbase4:37489] server.AbstractConnector(383): Stopped ServerConnector@49f3f2e8{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-24 20:11:23,792 INFO [M:0;jenkins-hbase4:37489] session.HouseKeeper(149): node0 Stopped scavenging 2023-07-24 20:11:23,793 INFO [M:0;jenkins-hbase4:37489] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@57548a1b{static,/static,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/classes/hbase-webapps/static/,STOPPED} 2023-07-24 20:11:23,794 INFO [M:0;jenkins-hbase4:37489] handler.ContextHandler(1159): Stopped o.a.h.t.o.e.j.s.ServletContextHandler@1e7ef621{logs,/logs,file:///home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/hadoop.log.dir/,STOPPED} 2023-07-24 20:11:23,795 INFO [M:0;jenkins-hbase4:37489] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,37489,1690229472216 2023-07-24 20:11:23,795 INFO [M:0;jenkins-hbase4:37489] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,37489,1690229472216; all regions closed. 2023-07-24 20:11:23,795 DEBUG [M:0;jenkins-hbase4:37489] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 20:11:23,795 INFO [M:0;jenkins-hbase4:37489] master.HMaster(1491): Stopping master jetty server 2023-07-24 20:11:23,797 INFO [M:0;jenkins-hbase4:37489] server.AbstractConnector(383): Stopped ServerConnector@13e4f5ac{HTTP/1.1, (http/1.1)}{0.0.0.0:0} 2023-07-24 20:11:23,797 DEBUG [M:0;jenkins-hbase4:37489] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-24 20:11:23,797 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-24 20:11:23,797 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1690229472739] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1690229472739,5,FailOnTimeoutGroup] 2023-07-24 20:11:23,797 DEBUG [M:0;jenkins-hbase4:37489] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-24 20:11:23,797 INFO [M:0;jenkins-hbase4:37489] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-24 20:11:23,797 INFO [M:0;jenkins-hbase4:37489] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-24 20:11:23,798 INFO [M:0;jenkins-hbase4:37489] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-07-24 20:11:23,798 DEBUG [M:0;jenkins-hbase4:37489] master.HMaster(1512): Stopping service threads 2023-07-24 20:11:23,798 INFO [M:0;jenkins-hbase4:37489] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-24 20:11:23,798 ERROR [M:0;jenkins-hbase4:37489] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 20:11:23,798 INFO [M:0;jenkins-hbase4:37489] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 20:11:23,798 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-24 20:11:23,797 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1690229472734] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1690229472734,5,FailOnTimeoutGroup] 2023-07-24 20:11:23,799 DEBUG [M:0;jenkins-hbase4:37489] zookeeper.ZKUtil(398): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-07-24 20:11:23,799 WARN [M:0;jenkins-hbase4:37489] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-24 20:11:23,799 INFO [M:0;jenkins-hbase4:37489] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 20:11:23,799 INFO [M:0;jenkins-hbase4:37489] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-24 20:11:23,800 DEBUG [M:0;jenkins-hbase4:37489] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-24 20:11:23,800 INFO [M:0;jenkins-hbase4:37489] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-24 20:11:23,800 DEBUG [M:0;jenkins-hbase4:37489] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-24 20:11:23,800 DEBUG [M:0;jenkins-hbase4:37489] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-24 20:11:23,801 DEBUG [M:0;jenkins-hbase4:37489] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-24 20:11:23,801 INFO [M:0;jenkins-hbase4:37489] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=54.62 KB heapSize=67.36 KB 2023-07-24 20:11:23,836 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:41283-0x10198f4f3ce0028, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 20:11:23,836 INFO [RS:3;jenkins-hbase4:41283] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,41283,1690229481649; zookeeper connection closed. 2023-07-24 20:11:23,836 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:41283-0x10198f4f3ce0028, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 20:11:23,837 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@715057ca] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@715057ca 2023-07-24 20:11:23,847 INFO [M:0;jenkins-hbase4:37489] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=54.62 KB at sequenceid=1058 (bloomFilter=true), to=hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/4c033d3d022d4dac8ee22f3dfa4c476b 2023-07-24 20:11:23,860 DEBUG [M:0;jenkins-hbase4:37489] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/4c033d3d022d4dac8ee22f3dfa4c476b as hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/4c033d3d022d4dac8ee22f3dfa4c476b 2023-07-24 20:11:23,866 INFO [M:0;jenkins-hbase4:37489] regionserver.HStore(1080): Added hdfs://localhost:37609/user/jenkins/test-data/cb660af9-a7f9-52dc-6c07-ec95ffdc8ea4/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/4c033d3d022d4dac8ee22f3dfa4c476b, entries=18, sequenceid=1058, filesize=7.4 K 2023-07-24 20:11:23,867 INFO [M:0;jenkins-hbase4:37489] regionserver.HRegion(2948): Finished flush of dataSize ~54.62 KB/55933, heapSize ~67.34 KB/68960, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 66ms, sequenceid=1058, compaction requested=true 2023-07-24 20:11:23,874 INFO [M:0;jenkins-hbase4:37489] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-24 20:11:23,874 DEBUG [M:0;jenkins-hbase4:37489] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-24 20:11:23,879 INFO [M:0;jenkins-hbase4:37489] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-24 20:11:23,879 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-24 20:11:23,879 INFO [M:0;jenkins-hbase4:37489] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:37489 2023-07-24 20:11:23,885 DEBUG [M:0;jenkins-hbase4:37489] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,37489,1690229472216 already deleted, retry=false 2023-07-24 20:11:23,936 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:33773-0x10198f4f3ce002a, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 20:11:23,936 INFO [RS:4;jenkins-hbase4:33773] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,33773,1690229483317; zookeeper connection closed. 2023-07-24 20:11:23,936 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:33773-0x10198f4f3ce002a, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 20:11:23,936 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@4a2b08bf] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@4a2b08bf 2023-07-24 20:11:24,537 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 20:11:24,537 INFO [M:0;jenkins-hbase4:37489] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,37489,1690229472216; zookeeper connection closed. 2023-07-24 20:11:24,537 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): master:37489-0x10198f4f3ce001c, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 20:11:24,638 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:46693-0x10198f4f3ce001d, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 20:11:24,638 INFO [RS:0;jenkins-hbase4:46693] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,46693,1690229472273; zookeeper connection closed. 2023-07-24 20:11:24,638 DEBUG [Listener at localhost/44145-EventThread] zookeeper.ZKWatcher(600): regionserver:46693-0x10198f4f3ce001d, quorum=127.0.0.1:62881, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 20:11:24,638 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@1d0b206c] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@1d0b206c 2023-07-24 20:11:24,638 INFO [Listener at localhost/44145] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 5 regionserver(s) complete 2023-07-24 20:11:24,639 WARN [Listener at localhost/44145] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-24 20:11:24,670 INFO [Listener at localhost/44145] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-24 20:11:24,777 WARN [BP-1903532105-172.31.14.131-1690229433619 heartbeating to localhost/127.0.0.1:37609] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-24 20:11:24,777 WARN [BP-1903532105-172.31.14.131-1690229433619 heartbeating to localhost/127.0.0.1:37609] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1903532105-172.31.14.131-1690229433619 (Datanode Uuid e3e2f909-b3c6-4182-9f15-560a4d523f97) service to localhost/127.0.0.1:37609 2023-07-24 20:11:24,778 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/cluster_82e812c0-1679-84b5-414c-93cfc125cc3f/dfs/data/data5/current/BP-1903532105-172.31.14.131-1690229433619] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-24 20:11:24,779 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/cluster_82e812c0-1679-84b5-414c-93cfc125cc3f/dfs/data/data6/current/BP-1903532105-172.31.14.131-1690229433619] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-24 20:11:24,781 WARN [Listener at localhost/44145] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-24 20:11:24,784 INFO [Listener at localhost/44145] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-24 20:11:24,888 WARN [BP-1903532105-172.31.14.131-1690229433619 heartbeating to localhost/127.0.0.1:37609] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-24 20:11:24,888 WARN [BP-1903532105-172.31.14.131-1690229433619 heartbeating to localhost/127.0.0.1:37609] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1903532105-172.31.14.131-1690229433619 (Datanode Uuid 3e40cac9-ace0-417b-a97f-3a6f56ab9729) service to localhost/127.0.0.1:37609 2023-07-24 20:11:24,889 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/cluster_82e812c0-1679-84b5-414c-93cfc125cc3f/dfs/data/data3/current/BP-1903532105-172.31.14.131-1690229433619] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-24 20:11:24,889 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/cluster_82e812c0-1679-84b5-414c-93cfc125cc3f/dfs/data/data4/current/BP-1903532105-172.31.14.131-1690229433619] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-24 20:11:24,891 WARN [Listener at localhost/44145] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-24 20:11:24,896 INFO [Listener at localhost/44145] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-24 20:11:24,999 WARN [BP-1903532105-172.31.14.131-1690229433619 heartbeating to localhost/127.0.0.1:37609] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-24 20:11:24,999 WARN [BP-1903532105-172.31.14.131-1690229433619 heartbeating to localhost/127.0.0.1:37609] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1903532105-172.31.14.131-1690229433619 (Datanode Uuid 32e0b698-571a-4a8e-8396-ca5cd6d8f1ca) service to localhost/127.0.0.1:37609 2023-07-24 20:11:25,000 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/cluster_82e812c0-1679-84b5-414c-93cfc125cc3f/dfs/data/data1/current/BP-1903532105-172.31.14.131-1690229433619] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-24 20:11:25,000 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-rsgroup/target/test-data/e0ba8734-2962-42f9-094a-8013d0891afe/cluster_82e812c0-1679-84b5-414c-93cfc125cc3f/dfs/data/data2/current/BP-1903532105-172.31.14.131-1690229433619] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-24 20:11:25,030 INFO [Listener at localhost/44145] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-24 20:11:25,157 INFO [Listener at localhost/44145] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-07-24 20:11:25,230 INFO [Listener at localhost/44145] hbase.HBaseTestingUtility(1293): Minicluster is down